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SUMMARY 




The planning study was one in which feasibility of proposed curri- 
culum plans in business and office education was evaluated by four cri- 
teria: (1) the plan was supported by professional leadership; (2) the 

plan evolved from available interdisciplinary thinking; (3) the plan 
provided a structure with profession-wide support and a feasible opera- 
tional structure; and (4) dissemination and determination of dissemina- 
tors, advocates, and acceptors were an integral part of the over-ail 
plan. 



Six drafts of a curriculum plan were developed before ail criteria 
of feasibility were considered met. Major problems met and conflict re- 
solution in reaching a defensible plan are reported. 

The plan accepted for a New Office and Business Education Learnings 
(NOBEL) uses an analog system model as a framework for development. The 
next step of the over-all project is the development of a set of educa- 
tional performance goals that represent business and office education- 
goals developed from hardware and software authorities of emerging of- 
fice occupations and from empiric data collected of office worker tasks. 

The professional structure now operative is Delta Pi Epsilon's Board 
of Governors for Research and Development. The Center for Vocational and 
Technical Education in agreement with the Board will continue as prime 
contractor, subcontracting with other institutions as deemed necessary 
to complete any future phases of the plan developed. 
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INTRODUCTI r 'J 



Moonshot. .is th r. business and office education curriculum project 
was first called, was initiated by the National Business Education As- 
sociation research Foundation, a department of the National Education 
Association, in October, 1966. Yet, a curriculum project with natirpwido 
impact needed support of more than this one organ { zaticn . A feasible 
plan would need support of the leadership from all organizations ano 
all segments of the profession. Thus, at a meeting of business and of- 
fice research leadership sponsored by NBEA’ s Research Foundation in May, 
1967, prior to the study herein reported, an ad hoc Business Education 
Study Committee (BES Committee) was formed to lead to representation from 
the totai profession in planning for curriculum renewal. 



M A Planning Study to Determine the Feasibility of Developing a New 
Business and Office Education Curriculum," developed by the BES Committee, 
is the subject of this report. In addition, the appendices contain capers 
commissioned and presented early in the study. . Their content provide 
interdisciplinary reaction to potential direction of a nationwide cur- 
riculum project such as Moons ho t and to application of a system approach 
and behavioral methodology in curriculum renewal. Available here for 
the first time, their contribution is to all those who would design or 
implement a new curriculum, whatever the field. 



Four Criteria of Feasibility Used in the Planning Study 

Any one of the teacher education leaders of the BES Committee could 
have developed a plan of action for curriculum renewal; but that any such 
plan of action must be determined to be feasible was also a condition of 
the study. ’’Feasibility," for our purpose, was defined in four ways. 

1. The first criterion of feasibility was that any plan of action 
developed had to be supported by most of the professional leadership. 

2. A second criterion of feasibility was that a plan of action would 

encompass available interdisciplinary thought wherever It could be found. 
Behavioral and other social sciences were viewed as sources of knowle ge 
and method that represented a gap as between what the business and office 
education curriculum is now and what it might become. The criterion re- 
quired the use of interdisciplinary thought from the social sciences: e.g. 

behavioral science and other learning theory, social psychology, economics 
and business administration, engineering (systems), communication sciences, 
educational technology, and research design and development. 

3. A third criterion of feasibility was one of an adequate organi- 
zational structure: a structure representing (a) a profession-wide 

leadership support of any plan of action developed; and (b) an operating 
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structure to implement it. 

4. The fourth criterion, the acid test of feasibility, was one of 
dissemination and acceptance. Dissemination has been an integral part of 
the planning study. Thus, steps were taken in the planning study to 
identify the various audiences who are the disseminators, advocates, and 
acceptors of proposed curriculum products. This last criterion required 
the continuation of these dissemination plans. 

The foregoing criteria of feasibility are inherent in the objectives 
as identified in the study proposal:* - 




1. Develop procedures for instituting a massive curriculum project 
in office occupations. 

2. Establish the organizational structure for administering the en- 
tire project. 

3. Develop the guidelines for the scope of the proposed curriculum 
project. 

4. Determine procedures for developing behavioral objectives: 

(a) review and assess the objectives and classification systems 
in current use for secondary educational programs 

(b) develop the criteria and procedures for building a complete 
catalogue of specific behavioral objectives for Moonshot 
curriculum program (grades 9-12) 

(c) develop a system for classification of the behavioral ob- 
jectives 

(d) validate the criteria, procedures and classification system 
by developing representative sub-sets of behavioral objec- 
tives. 

5. Develop procedures for communicating the Moonshot concept to the 
profession. 

6. Identify the talent available in the profession who will partici- 
pate in Moonshot . 

The planning study as herein reported thus required the development of 
a plan of curriculum renewal in business and office education determined to 
He feasible. The method of achieving this goal is the topic of the next 
discussion. 
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METHODS 



Operating; Framewo rk 

Feasibility of any plan of action proposed had to be determined 
within the operating framework designated by the BES Committee as follows: 

1. A steering committee from the BES Committee members was delegated re- 
sponsibility for implementation of initial stages of the. planning 
study to determine feasibility. (2) 

2 . The Center for Vocational and Technical Fducation at The Ohio State 
University was designated as the prime contractor. 

3. The principal investigator was named and operated through the Center 
by subcontract with The University of Michigan. 

Developing a Plan Acceptable to the Professional Leadership 

The first task required interdisciplinary reactions to the hunches 
suggested in the planning proposal: namely, that a systemic approach con- 

sistent with concepts of an organic curriculum (7) and based on behavioral 
theory and method could be employed in a curriculum action plan in business 
and office education. 

Within one week after funding (June 29, 1967), interdisciplinary con- 
sultants in the area of industrial relations and behavioral science, edu- 
cational psychology and technology, systems research and design, and edu- 
cational research and development design were retained with the following 
assignment: ,, 0n July 30 and 31, 1967, you will meet with representatives 

from business and office education in Chicago. You are asked to react to 
the proposed curriculum project as outlined in the planning study, criti- 
cizing its assumptions and hunches, and offering suggestions of direction 
which in your view such a project should take." Each of these consultants 
was located at the principal investigator’s university because of the limi- 
tations of time and resources. Yet, each was a national leader in his area 
of specialization and actively engaged in research and development in his 
field. 

For the principal investigator and his staff, the month of July was 
filled with meetings, group and individual, with the above consultants 
and with consultants retained by and with the Center staff. 

Two other behaviorl^ts were commissioned to make presentations in 
Chicago: Morrison (Appendix A), then director of the Quincy, Massachu- 

setts (3) curriculum project, and Canfield (Appendix B), then vice- 
president in charge of instruction at Oakland (Michigan) Community College. 
Oakland College was a newly developed thirteenth- and fourteenth-year level 
school with curriculum developed and taught following an educational sys- 
tems approach . ( 28 ) 
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In addition to the BES Committee and the Center's and principal in- 
vestigator's staff, representatives of Research Coordinating Units and 
state supervisors of business education were invited to participate in 
the Chicago meeting. 

Growing out of July 30-31 meeting, two four-day writing conferences 
were scheduled for August 5-8, and 12-15, 1967. The purpose was to dra r t 
a detailed plan of action for the total curriculum project. Two research 
design consultants were commissioned to write the plan with project per- 
sonnel and other individuals as available from the BES Committee. 

The draft from these writing conferences was reviewed by each con- 
sultant who had worked on some phase of the planning study, each person 
in attendance at the July 30-31 meeting, and special consultants called 
upon for the purpose by the Center (Appendix K-3). 

A special one-day review session with representatives of Federal and 
state supervisors was held at Columbus on August 22, 196/. Informal re- 
views by national research leaders were also asked for and received. 

The combination of feedback from all sources led to the following 
methodological decisions made early in September: 

1. An extension of time of the planning study frcm October 28 to Decem- 
ber 31, 1968 was requested. 

2. Revised drafts of the plan of action should retain the systemic ap- 
proach to total curriculum renewal. 

3. Interdisciplinary jargon of the first draft should be translated to 
language that could be readily understood by business and office edu- 
cators. 

4. While retaining the total system approach to curriculum renewal, only 
one phase of the system model should be attempted, the development of 
a set of performance goals for business ana office education. 

Thus, the last three months of 1967 were primarily times of writing, 
review meetings with Center staff and Center commissioned consultants, 
and rewriting. Six separate drafts of the proposal were prepared before 
general consensus of feasibility of the plan was reached. 

In the first two months of 1968, with the planning project extended 
further through February 28, development of data collection instruments 
and of training programs for investigators and performance goal writers 
was initiated. 

Developing an Organizational Structure 

The third criterion of a feasible plan of action, was developing a 
structure (a) representing profession-wide leadership support and (b) an 
operating structure. Of the two, the first was more critical requiring 
meetings to debate alternative structure and to strive for consensus. 
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Tlio criterion imposed a standard of agreement on structure, vhi rh , 
though not requiring unanimous consent, required more than a mere mao’*- 
Ity. Thus, two meetings held on structure resulted in but a slight ma- 
jority for organizing a nonprofit organization. The principal investi- 
gator prevailed with his steering committee to seek some other plan tlat 
would obtain greater support. 

After acceptance of the professional structure, develop. ng a con- 
st! tut lor and by-laws required additional meetings and dialogue. Added 
organization meetings were held to insure profession-wide reprr^rntat 1 * n 
and a delineation of operating policies which on the one hand would *n- 
surc professional responsiveness to plans of action and wh'ch on the 
othpr permitted a close liaison with all operating units. 

Developing an operational structure grew naturally out of that 
structure used in the planning study. The method for determining future 
operating structure was one of review and modification of that used in 
the planning study by professionals and operating personnel. 

Developing a Plan of Dissemination and Identification 

The fourth criterion of feasibility related to dissemination and 
the development of a plan for identification of disseminators, advocates, 
and acceptors. By method of determining, this criterion was the only 
one that could be separated from the mainstream of the investigation. 

Thus, determination of the first three criteria of feasibility was the 
primary responsibility of the principal investigator. The fourth was 
separated with budget retained at the Center for dissemination and funds 
allocated to personnel at Wayne State University for a plan of identifi- 
cation. Details of methods used and their results are contained in Ap- 
pendix G and H. 

Thus, the specific methods of the planning study have been described. 
Only the specificity of detail modified slightly the general outline 
method originally proposed. (4) As a summary of method employed, the gen- 
eral outline as then stated follows: 

1. To convene a panel composed of personnel with the following 
competencies, research design, curriculum design, behavioral 
science, business administration, business education, and sys- 
tems design to develop procedures for the project. 

2. To secure adequate representation of the total profession and 
all the fields that impinge on the proposed project: 

(a) preliminary steps have been taken through the organization 
of the Business Education Study Committee by leaders in 
the field. 

(b) further steps will be taken to form a nonprofit corporation 
whose sole objective is to carry out the Mo on shot curricu- 
lum project. 
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To convene a group composed of selected teacher educators, super- 
visory personnel, executives, and classroom teachers to secure, 
recommend, and to develop the parameters of the total project. 

4. To secure a behavioral scientist to: 

(a) review the literature pertaining to the development of be- 
havioral objectives and have him produce guidelines for 
use in office occupations; 

(b) use these guidelines for writing behavioral objectives for 
the project. 

5. To have a position paper developed on the recommended dissemina- 
tion procedures to: 

(a) disseminate the Moonshot concept to our audiences: teacher 

educators, state and local supervisors, curriculum and 
other specialists; 

(b) to use the position paper in developing a pilot project to 
inform in-service teachers about the scope and content o 

Moonshot ; 

(c) to use the position paper to prepare materials for keeping 
city and state supervisors, RCU directors and other members 
of our audiences informed of the project. 

6. Identify the talent available in the business and office educa- 
tion profession who can participate in the Moonshot. project: 

(a) develop a roster of selected personnel: city supervisors, 

state supervisors, business teacher educators, and others; 

(b) secure "talent data by means of a mailed instrument to the 
above (see Appendix G for a suggested talent inventory in- 
strument); 

(c) develop a computer program for analysis of this talent 
available for project Moonshot . 
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RESULTS AND FINDINGS 



From tho current study, curriculum renewal in business and office 
education has been deemed feasible and a plan for implementation has beer, 
drawn. (i6> The major feature of the plan is a process of renewal which 
is inherent in the specifications of an analog system model. Future plans 
for curriculum renewal are based on this systemic model. Yet, to describe 
the result of the study as a systemic model without also describing the 
time-consuming, tedious, and frequently frustrating process of conflict 
resolution needed to derive the present status would distort the findings. 

The method of planning to determine feasibility imposed the seeking 
of interdisciplinary viewpoints. It imposed the exposure of any tenta- 
tive plans derived to the critical appraisal of many segments of the pro- 
fession. It imposed the critical scrutiny of the BES Committee, 21 
scholars who would naturally subject any proposition to which they must 
later subscribe to critical appraisal (5). 

The diversity of thinking provided a continuing round of problem 
solving. Each problem, though not always answered adequately for the one 
who proposed it, required on the part of the principal investigator staff 
an acceptance, reassessment, rewriting, debating, and writing again to 
the point that resolution at least permitted accommodation. 

In retrospect, the crux of feasibility later determined, was thr 
early acceptance of the systemic model of curriculum renewal, a process 
model. The "New" Office and Business Fducation Learnings (NOBEL) ana- 
log system model is reproduced on the next page. The emphasis on pro- 
cess rather than product turned put to be a most important means of con- 
flict resolution. While men of goodwill might not resolve all their di- 
verse views cn the product sought, they could and did agree on a process 
for achieving it. 

NOBEL System Model as a Process 

Theoretically, the NOBEL system model accommodates differing philo- 
sophic values, values that serve in place of perfect knowledge. The feed- 
back circuitry of NOBELS provides closure making the system self-correcting. 
Whatever values tested through the system, the resulting feedback and 
modification should lead at least a step nearer to perfect knowledge, 
thus lessening the gap among differing values. 

The NOBEL system process requires specification of values to be 
tested in design criteria which, if not measurable, are at least asses- 
sable in terms of over-all performance of the system. The NOBELS model 
further demands the specification of the immediate set of expected learn- 
ings stated in behavioral terms. The means of attaining the expected 
learnings are defined in terms of persons, processes, or properties as 
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’’input” or application. The application yields ’’output” of learning 
which must be assessed in terms of performance of the learner. The as- 
sessment of output supplies the feedback circuitry with data necessary 
for analysis and modification. Feedback circuitry thus provides for 
closure of the system. The analog model for NOBEL is described further 
in Appendix I. 



Initial Limitations 

In a practical way, NOBELS was limited to business and office oc- 
cupations education. Yet, the system tryout could just as well be the 
broad area of business and economic education, distributive occupations 
education, any other vocational area, or any phase of education in gen- 
eral. The focus on office occupations as the starting point was dictated 
by the educational background and experience of the original Committee. 

The generality of NOBELS model has, however, been the impetus for initial 
parallel planning in the broader area of business-economic education. 

It has also motivated inclusion of distributive educators in the organi- 
zational structure and close collaboration with all areas of vocational 
education. 

The first product viewed for curriculum implementation, a set of 
performance goals that represents business and office education, may well 
be input to a combination of general education goals consistent with or- 
ganic curriculum concepts. (7) Such a use will, of course, require a dif- 
ferent organizational structure than one limited to vocational or, as now, 
a limited area of vocational education. The ability of NOBEL system to 
be used later as a sub-system input for a broader educational purpose is 
as least face validity of the flexibility inherent in the model. 



’’Purists” Versus ’’Developers” 

From the beginning, initiators as individuals and as groups have 
recognized ’’research impurity” of the development- type project of cur- 
riculum renewal. However, the BES Committee (5) supported the develop- 
ment process as a practical means of influencing classroom practice in 
the immediate future. While the NOBELS model accommodates ail applic- 
able ’’test-tube" findings, it further invokes a test of application of 
those findings in "real” learning situations where many variables may 
not even be identified, let alone controlled. The support ot the pro- 
fession for the development- type project recognized the immediacy for 
curriculum renewal, an immediacy that cannot await test-tube purity of 
knowledge. 



Tabula Rasa 

Carpenter (Appendix D) suggested capturing "good" current programs 
as a beginning test of the systemic model. Unfortunately, "goodness" 
in current business and office education programs has not generally been 
defined in specific and assessable terms. Even those programs of re- 
puted goodness would have had to be analyzed in terms of their presumed 
performance objectives and their outputs reassessed in those terms. The 
"clean slate" decision made recognizes the office occupation (current 
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an«i emerging) as a source of potentially valid performance goals for 
learnings programs, goals that can be deduced directly from office tasks 
rather than from often dimly viewed goals of some current programs an^ 
whose validity remain in doubt. 



Analog Versus Mathematical Model 

Wilson (Appendix C) discouraged business and office educators frem 
professing to a systemic approach to curriculum renewal because of the 
imprecision of any first attempts and the absence of resources to attain 
a mathematical preciseness. The NOBEL system model developed primarily 
by Hill (8) recognizes that any learnings model will he imprecise now 
and perhaps oversimplified. Yet, the model does invoke assessment and 
feedback circuitry providing seif-correction— a necessary first of many 
steps which may eventually lead to mathematical simulation. 

The danger of oversimplification in the current model has, of course, 
been recognized. For example, the model does not reflect important inter- 
faces of the curriculum with the business office or of impinging legal 
authority such as local, state, and Federal authorities. The model does 
not reflect adequately the subordinate position of business and office 
education as a part of a total educational system. The model may also 
inadequately depict subelements of the system such as the student, the 
object of ail learnings programs. 

Even with recognized limitations, though, NOBEL does provide a sys- 
temic framework which can be understood and thus embraced by classroom 
practitioners, a requisite later to advocacy and adoption of any curricu- 
lum product. 

Communication Gap 

Failure of translators as among disciplines often leads to lack of 
understanding or misunderstanding. First statements of the systemic 
model were sometimes clothed in the jargon of those steeped in the tech- 
nical vocabulary of systems design. Misunderstanding and dissatisfaction 
resulted among professionals in business and office education. Transla- 
tion to nontechnical language became a necessity. Further simplifica- 
tion and standard meanings of terms is recognized as a continuing prob- 
lem as classroom dissemination and advocacy proceeds. One such attempt 
to simplify interdisciplinary ideas is contained as Appendix L. While 
the loss of disciplinary depth is not here assessed, the acceptance of 
even approximations of thought from other disciplines by business and 
office educators depends upon such translation. 



Experiential Validity Versus Thomism 

By "experiential validity," we mean the determining of business 
and office learnings based on tasks that can be observed or experienced 
in "real” offices. By Thomism, we mean a theoretical framework that pro 
vides the order and method of a disciplinary approach to learning. 
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The conflict of functionalism versus a disciplinary approach is not 
uniaue, of course, to business and office education. Its current emer- 
gence as a problem results from recent technological advances in infor- 
mation handling and consequent development at collegiate levels of com- 
mvmication or information sciences. 







Out of the hardware and software technology has developed thinking 
that would equate office work with portions of information science. The 
idea is that such a parallel provides in information science a theoreti- 
cal base for analyzing business and office occupations education and thus 
for developing a "discipline” for business and office education. 

The preoccupation with theory building to provide "a new way of 
looking at office occupations" can be recognized in the original planning 
and feasibility proposal. Here, various Venn diagrams were used to iso- 
late office work as a means of leading to the theoretical framework. (9 ^ 

A three dimensional analysis scheme similar to Guilford’s earlier "Three 
Faces of Intellect," (lO) was also proposed. ( 1 1 ) 

In the educational area, Krathwohl’s and Bloom’s taxonomies (12) 
were thought provoking as were the work of behaviorists like Gagne. (13) 
The affective domain of Bloom has served to awaken the educational com- 
munity that learning must be concerned with more than the cognitive. 
However, that the affective domain can be or even should be separate from 
the cognitive in office-task learning is not clearly seen at this time. 
Gagne’s Conditions of Learning will prove helpful later as office tasks 
are arranged in a learning order. 




The difficulty with utilizing earlier conceptualizations as the 
foregoing in the planning and feasibility proposal, intriguing as they 
are, has been one of demonstrating their validity in the "real-live" 
world of office work or their practicality in developing new business 
and office education learnings programs. 

Motivated, however, by these earlier conceptualizations, the staff 
at the Center for Vocational and Technical Education has developed a 
three dimensional scheme for classification of office activities (verbs) 
according to managing, operating, and interacting domains. (14) The tax- 
onomy as a means of classifying empiric data collected from offices does 
provide an immediate and useful tool* That the taxonomy will, in fact, 
provide a "new way of looking at" or analyzing office work, the desired 
theoretical framework, remains to be proved. Yet, this systematic clas- 
sification scheme could well be a first step toward the "new look." 

Another problem related to developing a theoretical framework is the 
extent to which sets of office tasks, even valid in terms of those tasks 
performed in the "real" office, teach the methods of "learning to learn," 
problem solving, or adjusting to novel situations— characteristics of of- 
fice workers needed for adaptation to changing tasks. Perhaps the methods 
of information science, i.e., system analysis, or methods used by the 
accountant to analyze financial transactions must be taught apart from 
the office tasks to be learned. Behaviorists lean toward a view that 
problem solving is inherent in the tasks necessary to be learned— a view 
that needs further study. 
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' Total Versus Partial Systemic Testing 

Through interdisciplinary thinking, a first draft of NOBELS fol- 
lowed a pilot testing plan of the total systemic model. In this draft, 
a limited set of learning objectives were to be selected from business 
and office occupations; their means of learning (input) through persons, 
processes, and properties developed or adapted; their learning results 
| (outputs) tested; and modification of the system or its *irst output 

! made in terms of analysis. The system thus set in motion would then he 

applied to increasing segments of office education. 

: This first NOBELS draft was rejected for two reasons: (1) the sys- 

; tem jargon used in writing the plan was confusing to reviewers from busi- 

ness and office education; (2) the foreseeable resources did not match 
the expected cost of even the pilot, some $2-million dollars. 

While retaining the system model, a second draft limited initial 
steps of curriculum renewal to the development of a set of performance 
goals that represent business and office education. The absence of any 
such comprehensive set of goals had been noted, and their provision in 
specific behavioral language could lead to a counterbalance for the cur- 
rent overemphasis on classroom methodology. In fact, the specification 
f of behavioral goals in a classroom situation could contribute to method 

| by insuring teacher and learner of direction, often but vaguely discem- 

[ able in classrooms today. Advantages of the second plan over the first 

are at least four-fold: (1) the tasks necessary to developing a set of 

performance goals is within the foreseeable resources of the investigators; 
| (2) adaptation of the set of goals to classroom learning can be made im- 

mediately; (3) decisions about ways of using and testing the set of goals 
further in development programs can be delayed; (4) parallel curriculum 
■ programs can be formulated before the completion of the plan. 

In the reporting thus far, the current status of the plan of action, 
NOBELS, some of the major problems determining NOBELS* feasibility have 
j been described. Organizational problems were also important in deter- 

t mining feasibility, the next topic. 

i 

\ 

c 

i 

l Organizational Problems 

I 

Another important set of time-consuming problems was those related 
[ to developing an organizational structure of the profession. The re- 

sulting structure has been detailed elsewhere. (15) In brief, the struc- 
ture developed does represent all segments of the business education pro- 
I fession. It is one organized as a part of Delta Pi Epsilon, national 

honorary graduate fraternity, and called H DPE*s Board of Governors for 
f Research and Development" (Appendices K-l and K-2). This entity has been 

I organized as one to evaluate, to propose or to review proposals, and to 

l seek funding for all proposals considered worthy of profession-wide sup- 

port, 
f 

| That such an entity does now exist attests the perseverance, energy, 

and goodwill of many leaders. The bias stated in the planning study pro- 
posal (l6) was a nonprofit organizational structure separate from existing 
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organizations. A simple majority of the BES Committee favored the non- 
profit organization. A large minority viewed organization of a nonprofit 
organization as a proliferation of entities. Against conflicting advice, 
the principal investigator did not accept a simple majority as authoriza- 
tion for organizing a separate nonprofit entity. Instead, a BES Subcom- 
mittee was appointed to seek a structure that could be supported by most 
of the larger group. Delta Pi Epsilon tendered an invitation to organize 
as DPE f s Board of Governors for Research and Development. Upon recommen- 
dation of the BES Subcommittee, the invitation was accepted with but one 
dissenting vote. 

In tendering the invitation, DPE indicated a desire to provide an 
institutional home without line control of the Board 1 s activities. Such 
a loose arrangement of control by DPE was feasible and desirable because 
of overlapping leadership in both groups. Such an institutional home was 
desirable because DPE as a national honorary fraternity in business edu- 
cation is the one organization to which all segments of the profession 
can and do identify. 

With a feasible structure assured, a series of meetings was necessary 
to plan the internal organization — again, time-consuming but necessary to 
discuss issues and reach consensus on constitution and by-laws of the 
Board. The attainment of the Board of Governors as the structure met one 
important criterion of feasibility as defined. 

The structure accepted for operational implementation of plans is the 
Center for Vocational and Technical Education who will serve as prime con- 
tractor for future funded projects as mutually agreed upon by the Board 
and the Center. Major portions of any contract are now planned to be sub- 
contracted to other educational institutions. For example, the principal 
investigator for a proposed step of NOBELS will be located at Wayne State 
University (17) and the subcontract for his responsibilities will be one 
drawn as between the Center and that University. 

Growing out of experience in the planning and feasibility study, the 
Center has now appointed a NOBELS Executive Committee as a policy making 
and review arm of the Center. Thus, the Center* s NOBELS Executive Com- 
mittee becomes the major group to propose policy, review and advise about 
work done at the Center and at the principal investigator* s or other sub- 
contracting institutions. The absence of such a policy group in the 
planning and feasibility study did lead to some confusion and, at times, 
duplication of effort. Such a policy group which clearly delineates 
responsibilities is a necessary control for the Center as well as a neces- 
sary link with leaders in business and office education. 



Plan for Developing Behavioral Objectives 

The decision that initial steps of curriculum renewal should involve 
development of a set of performance goals representing business and of- 
fice education provided another series of intriguing problems to be solved. 
Behaviorists specified procedures for developing performance goals and re- 
levant literature for their development. (18) 
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Experiences of Morrison (Appendix A), Canfield (Appendix B), and 
Rummier (Appendix F) first identified the difficult problem of training 
classroom teachers to write relevant performance goals. Carpenter (Ap- 
pendix D) also emphasized the critical problems of teacher education. 

The principal investigator using Mager,(18) Vimcet Filmstrips, (18) and 
Hite (19) has experimented informally with teaching performance-goal 
writing to practicing teachers, graduate students in his classes. In 
addition, Rummier and his associates have worked with the principal in- 
vestigator staff in teaching methods used at the Center for Programmed 
Learning for Business of deriving critical behaviors, a foundation for 
performance goal writing. Further experimentation is needed and planned 
for the Summer, 1968, in developing a training program for performance 
goal writers. The tentative conclusions are these: (1) performance goal 

writing for the initial set of objectives representing business and of- 
fice education will need to be developed and/or reviewed by leadership 
personnel in the fieldf (2) performance goal writing for NOBEL should not, 
however, be divorced in any important way from the business and office 
education investigators; (3) simplification of teaching of performance- 
goal writing and amplifying thexr values by teacher educators is pre- 
requisite to advocacy and acceptance by classroom teachers. 



Problems of Task Performance Data Collection 



To provide a balance between the "oughts" and the "is's" of the set 
of NOBEL goals developed, the emerging office occupations are in process 
of being analyzed as an important input of "ought- to-be” performance 
goals. (Appendix H-l and H-2), Empiric data of existing office tasks 
(the is's) are planned to be collected in nine population census regions 
of the United States. (20) 

Judgmental sampling decisions have been made for proposed (20) data 
collection that approximate U. S. Census statistics (2l) of office work- 
ers according to proportion found in certain Standard Industrial Code 
classifications businesses, in geographic locations and in size of com- 
panies, as well as according to sex. 

A gradual development of performance goals and task lists prior to 
major data collection is viewed as one means of decreasing the redundancy 
of handling data collected. Yet, the frequencies of office task perform- 
ance, their conditions and contingencies, and their varying standards of 
performance will provide significant data for determining relative impor- 
tance of different tasks. A coding scheme tied to the new "Taxonomy" of 
Office Activities (14) will serve to classify like task data. Prophecy 
formulas applied to pilot data, however, may lead to a reduction of em- 
piric data now planned to provide an adequate sample. 

Data-collection decisions centered around Dictionary of Occupational 
Titles (22) (DOT's) versus some other scheme presented one interesting 
problem. (20) Even though DOT'S and their accompanying descriptions are 
the best source of job classification information, census sources desig- 
nated (21) at least one-third of office jobs as "not elsewhere classified." 
Also, the "average" descriptions of specific jobs contained in DOT'S do 
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not necessarily match the range of tasks performed in specific office 
jobs. Furthermore, the overlap of task performance as among office jobs 
has been observed. (23)(24). For all these reasons, the decision was 
made to select office jobs for data collection without reference to their 
DOT or any other classification according to job title. 

The decision to study jobs held by people between the ages of 16 and 
24 year olds (20) was motivated by the need for collecting data from a 
hierarchy of jobs held by a range of young people. The age range reflects 
the need for those data to be relevant to tasks subject to inclusion in 
learning programs of secondary, area vocational, and community college 
level institutions. 



Specific Job Task Data Collection Problems 

A variety of methods for collecting relevant data from a specific 
job has been explored and evaluated in terms of economic efficiency and 
yield of relevant data. Observational techniques, time sampling tech- 
niques, photographic and film techniques, critical incidents, group inter- 
view, employee interview, supervisor interview, work samples, and combina- 
tions thereof have been examined. A combination of job diary plus employee 

and supervisor interview is now planned as the source of empiric job data 

yielding within one-half day of data collector's time the basic informa- 
tion needed for performance goal preparation. (20) The specific instru- 
ments and procedures are in process of being developed. 



Social Role Data Collection 

Through the influence of research in social psychology, a promising 
type of data appeared at first to be the collection of social role data, 
normative or expected, of office workers. (25) Further study, however, 
does not support the collection as originally planned. To be sure, be- 
havior of the individual office worker in task performance involves in- 
teraction with people; but the interaction with other people and its con- 
sequent coping problems is now viewed as an integral part of task per- 
formance. Separation of social role behavior from any other task behavior 
is not now considered a promising direction. Social role behavior will 
be collected as an integral part of task performance. 



Dissemination and Talent Inventory 



Originally, the planning study proposed commissioning a position 
paper on recommended dissemination procedures. However, early in the 
study, the high priority of determining a theoretical framework as a 
foundation for restructuring business and office education was recognized. 
The Center»s efforts were diverted, therefore, to this task. A discussion 
of activities in the planning study as conducted at the Center is con- 
tained in Appendix H. 



The Talent Inventory, i.e., identifying groups of disseminators, ad- 
vocates, and acceptors was an integral part of the planning study performed 
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at Wayne State University. The original plan was extended to include, 
and now includes a 60 percent completion of the foliow-up of the talent 
inventory collected in the planning study and a 50 percent completion 
of an updated inventory of doctoral candidates. A detailed report of 
the talent inventory conducted through the planning study is in Appendix 
G. 



Other Results 



Throughout the planning study, a continuing responsibility has been 
that of developing liaison with other on-going curriculum development 
projects from which methodology could be borrowed or results incorporated. 
Frequently, published reports or first-hand accounts were sufficient. 

For example, the principal investigator of the Quincy, Massachusetts, cur- 
riculum study (3), formerly located at the American Institutes for Re- 
search in Pittsburgh, moved to the Center for Vocational and Technical 
Education as Research Coordinator. He thus provided first-hand informa- 
tion about this study. Other curriculum projects required visitation, 
and the following discussion is suggestive of the results of these on-site 
visits. 

Within the geographical area of Michigan, visits have been made to 
Michigan State University's block- time office curriculum project (26) and 
Wayne State University* s senior intensified program. (27) In both pro- 
jects, innovative curriculum means are being demonstrated, based on cur- 
rent curriculum goals. As an opinion, each demonstration of learning re- 
sults can, in part, be attributed to a Hawthorne effect on teachers and 
students, an effect that could well be incorporated in all learning pro- 
grams. Both projects could profit from a set of specific behavioral ob- 
jectives outlined as the next step of NOBELS. 

Oakland Community College represents a pioneering educational effort 
in which all curriculum subjects, including distributive and office edu- 
cation, are based on an instructional systems approach. (28) A pioneer 
must overcome the bias of tradition among teachers. That Oakland Com- 
munity College will be successful in its curriculum approach at this 
level of education is a hoped-for result. In the meantime, this systemic 
approach has influenced and is influencing community college curriculum 
in Michigan and other parts of the country. 

Monroe (Michigan) Public School is an ES70 (29) school-designate for 
developing a systemic approach in general business and vocational distribu- 
tive and office education. As yet, the development of performance goals 
in these areas has not gone beyond the planning stage. A most pressing 
need expressed by the faculty is that of a workshop to learn to develop 
behavioral goals. 

Through the Research Coordinating Unit of Michigan, the principal 
investigator visited Lansing (Michigan) Community College. Individualized 
instructional courses have been and are being developed utilizing autoin- 
structional guides and teacher developed single-concept films and film- 
strips. 
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Dade County (Florida) Public Schools, (30) although not an ES70 
designate, was visited because of its organic curriculum development 
project. A member of the Dade County Instructional Research team was a 
research associate in business education. In addition, innovative in- 
structional means (e.g., programmed instruction, large group instruction 
in typewriting) were under development. A two-day visit to Dade County 
dispelled the concern that these efforts were parallel to the current cur- 
riculum plans. A coding scheme for behavioral objectives developed in 
Dade County will prove helpful in solving similar problems for education- 
al purposes of office activities, office tasks, and emerging or current 
performance goals. 

McCloskey, Perkins, Byrd, et «1. were reported in process on a pro- 
ject titled ’’Knowledge and Skills Required for Clusters or Families of 
Occupations” (31a, b, c) including one study of empirical data collected 
for office occupations in the State of Washington. Their study is sug- 
gestive of the data collection problems to be encountered in NOBELS. Yet, 
NOBELS will modify data collection in terms of job tasks that can be stated 
in behavioral terms. By way of serendipity, Herbert Hite’s report, (1 ) 

"An Experimental Model for Preparing Objectives,” obtained while visiting 
Washington State University at Pullman provided an important baseline pro- 
ject for a training program needed to develop similar performance goal- 
writing competency in the next stage of NOBELS. 
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CONCLUSIONS AND RECOMMENDATIONS 



The planning study involved developing a plan for a New Office and 
Business Education Learnings System (NOBELS). An analog system monci was 
developed as the framework in which NOBELjcould be developed and tested. 
The next step in NOBELS was determined as developing a set of performance 
goals to represent business and office job tasks. (6) 

Four criteria of feasibility were applied to the plan with these 
results: i. NOBELS plan is supported by the professional leadership, 

both those who are members of DPE's Board of Governors (Appendix K-2) 
and by other business and office leaders who were asked to review the 
plan. (Appendix H-i) 2. Interdisciplinary thinking was used throughout 
the development of NOBEL plan (Appendix K-3). J. A structure of pro- 
fessional organization ana of operating units is functioning (Appendix 
**• Plans of dissemination and identification of disseminators, 
advocates, and acceptors are continuing (Appendix G). 

The most pressing problems and conflicts met and resolved were, pri- 
marily those requiring face-to-face meetings with adequate time for 
understandable communication, review, debate, and revision. Six drafts 
were written before all criteria of feasibility were determined to be 
met. 



One advantage of a developmental project as NOBELS is that immediate 
recommendations are determined and often modified by preceding steps just 
concluded. Long-range recommendations, however, cannot be structured in 
any iconic form— except as a process for successive approximation toward 
the pictured result. The analog system model of NOBEL is such a process 
model admitting to closure and thus analysis and system modification 
through feedback about currently viewed results. We plan that each phase 
or movement toward the curriculum project will utilize the NOBELS model 
as a process of continuous curriculum renewal. We recommend further that 
system feedback be employed at each stage of th total project to modify 
as needed the NOBELS model more nearly to clarify the process of curricu- 
lum renewal. We suspect, for example, in its currently oversimplified 
form, the system model does not now adequately depict important inter- 
faces of students, of teachers, of other educational systems of which 
business education is a subsystem, or of the changing business world. 

In terms of individual students, any operational learnings resulting 
from performance goals will need to be modified in terms of pupils, their nature 
and nurture. While the best hunches of learning theorists employed to 
describe students must be utilized in a first tryout of a new learning 
set, these educated guesses, though an improvement over current practice, 
leave broad areas of ignorance about pupils and the way they learn. * 
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In terms of teachers, a movement away from traditional, classroom 
practices presents to many of them a threat not readily overcome. A role 
chance of the teacher, for example, from a traditional ’’giver of informa- 
tion” to a "manager of learning" is fraught with obstacles requiring a 
total educational system to overcome. In fact, the reluctance of Public 
educators to move toward a systemic means where the product in learning 
can be assessed has been a prime motivation for other institutional 
structures to fill the breach. 

In the area of performance goal development, the next stage of NCBCT3, 
initial sets of goals will need to be created, demonstrated, and advocated 
by leadership from' all teacher education and behavioral fields. Yet, if 
the NOBEL systemic process is to become a self-renewing tool, means must 
be found for all classroom practitioners not only to learn how to develop 
performance goals in behavioral terms but to value their development as 
the foundation of curriculum and practice. Thus, we recommend that ef- 
fort be expended in the next stage of the business and office curriculum 
project to develop training programs for performance goal development 
that promise to lead practitioners to competency in preparing and to 
value in using behavioral goals. Immediate dissemination of even interim 
results should be made to all teacher education institutions. 

Attention of all areas of vocational and general education is called 
to the NOBEL systemic model developed through this planning study. We 
especially urge parallel projects be implemented in general business- 
economic, preparatory, and vocational distributive education areas. As 
the interface of business and office education to all areas of education 
and the total system of education is more clearly defined, the necessity 
for broad interdisciplinary thinking in all curriculum development as 
employed in the planning study will become even more apparent. 

Vocational business and office education is unlike areas of general 
education whose curriculum reality is usually a compromise as between 
Thomism as dictated by collegiate disciplines and experiential validity 
represented by Deweyism. The validity of vocational office education 
is directly related to current or emerging tasks performed in the "real” 
business office. Though frequently studied, the interface of business 
to business and office education has been poorly understood. We recom- 
mend therefore, that the interface as between business and business edu- 
cation in the next stage of NOBELS be more clearly defined. 

The impact of Moonshot or NOBELS on the total business and office 
education profession has not been assessed in the planning study in any 
formal way. As a subjective judgment, leadership represented by re- 
searchers, business educators, and all levels of supervision recognize 
the necessity for NOBELS. The leadership is receptive to change and will, 
of course, become the advocates when change is recommended. 
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APPENDIX A 



SOME PROBLEMS IN DEVELOPING CURRICULA 
BY ANALYSIS OF PERFORMANCE OBJECTIVES 1 

Edvard J. Morrison 2 
American Institutes for Research 



It is a significant forward step to decide as you have that a cur- 
riculum will be based upon analysis of the performance capabilities de- 
sired of graduates. I would argue that this is the proper basis for all 
curriculum development. But many who are less convinced than I of the 
validity of the general proposition would agree with you that specifica- 
tion of terminal performance capabilities is essential to development of 
relevant, effective, efficient vocational curricula. 

For some time now, the American Institutes for Research has been en- 
gaged with public school people in several projects to develop curricula 
which are relevant to three kinds of student needs: vocational competence, 

responsible citizenship and continuing self-fulfillment. The curricula 
are Intended to permit individually prescribed sequences and rates of 
learning, to challenge each student to achieve capabilities of his own 
choosing, and to develop the skills students will need throughout life 
as they evaluate, select and pursue educational and vocational routes to 
life goals. The graduates we seek are competent in all three of the broad 
areas of life and flexible enough to adjust successfully to changing ex- 
ternal requirements and personal needs. 

The analytic-empirical procedures we have used to develop curricula 
from such general goals as vocational competence, responsible citizenship 
and self-fulfillment are described in detail for one project in the re- 
ports listed at the end of this paper and will not be recounted here. How- 
ever, it should be noted that our procedures, as those you propose, were 
designed to produce a curriculum defined by a sequenced set of objectives 
stated in terms of the performance capabilities to be demonstrated by suc- 
cessful students. Using empirical information on what competent people 
must be able to do, our plan has been to arrive at the final set of ob- 
jectives by identifying the whole domain of capabilities implied by each 
general goal, selecting those capabilities to be Included in our curricu- 
lum, and then analyzing the selected terminal capabilities to define se- 
quences of prerequisite capabilities. The curriculum must be stated in 
terms of the things students must learn to do so we can have an adequate 
basis for selecting the materials and activities to facilitate learning 
of each objective, for defining methods by which achievement of each ob- 
jective can be verified, and for sequencing the learning activities. 



* ^Sailed on one section of an address to a meeting of educators pre- 
paring a proposal for a new office and business education learning system 
(NOBELS) in Chicago, 30 July 1967. 

2 Now at The Center for Vocational and Technical Education, The Ohio 
State University. 



In our attempts to develop curricula in the public schools by this 
plan, we encountered some problems. None of these problems has caused 
us to change our minds about the correctness of the development pro- 
cedures, but we now have a much better understanding of the kinds and 
amounts of difficulty associated with executing the procedures and using 
the methods. This should be useful information for others, such as your- 
selves, who would proceed in similar ways. So, my purpose is to tell 
you about a few of our major problems and invite yea to benefit from our 
experience . 

Objectives 

It has not been easy in my experience to get good performance ob- 
jectives in any area and I think you will find it to be difficult in 
business education too. Many people object to performance objectives 
because they believe their subject's content cannot be "reduced" to be- 
havioral statements since it deals with "understandings," "appreciations," 
perceptions, and the like. No matter how hard you argue that such 
things derive their importance from their effects on students' abilities to 
do things, many people with whom you wish to work will find it impossible 
to define objectives in behavioral terms. Even those who can accept the 
concept of performance objectives often find that it requires a kind of 
thinking to which it is difficult or impossible for them to switch. Fur- 
ther, it is difficult to get consensus among subject-matter specialists 
on the definition and selection of performance objectives, especially in 
complex areas and in areas traditionally considered "thinking" areas, 
such as English and Social Studies for example. 

So, I would caution you first of all that performance objectives are 
not easily obtained, though it is very easy to talk about them glibly. It 
is difficult to do the special kind of very, clear, objective thinking re- 
quired in preparing unambiguous objectives and it is easy to write objec- 
tives which are uncertain and obscure. Probably, you should plan on 
several cycles of writing- testing-writing en route to a set of fully ef- 
fective objectives and their supporting learning materials and programs. 

General Cap ab j 1 1 1 i e s 

You should be encouraged in your plan to search for general izable 
capabilities and to include them in your business and office curriculum 
because the efficiency and power of the curriculum will be enhanced by 
their inclusion. However, you also should be forewarned that this com- 
monly has been a more difficult undertaking than it appeared to be on 
first consideration. Many argue that we must train people so they will 
be prepared to "roll with the punch." No matter what happens in the labor 
market, no matter what happens in life situations, no matter what happens 
in the political world, these people would be prepared because we taught 
them the basic skills, the things which are general, the things which are 
generally applicable. But we have not found it easy to identify such 
generalizable capabilities or to teach them. If there are generally ap- 
plicable capabilities, then it is obvious that we should teach them to 
everyone. The only questions are, "what are they?" and "How do you train 

for them?" On the answers to these questions, there is almost no agree- 
ment at all. 
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Ther© are numerous traps into which one may fall in trying to iden- 
tify generalizable capabilities; let me illustrate two of them. First, 
some people refer to "general background material" and claim this to be 
general preparation. This use of "general" typically means "remote from 
roility, in the sense that what is learned is expected at best to con** 
tribute indirectly and with unknown probability to the learner’s ability 
to deal with his life situations. Presumably, your decision to derive 
curriculum content from the performance capabilities desired of graduates 
was based on the intention to avoid content of such dubious relevance 
to students needs. In that case, this trap can be avoided by defining 
general capabilities to be those which are demonstrably useful in a wide 
variety of life situations of consequence to the student. Do not think 
of general capabilities as vague or remote from reality, 

A second kind of trap is illustrated by those who say that there 
are many general capabilities in vocational education and produce a list 
which includes things of which blueprint reading and welding are fair 
examples. So far as I can tell upon reasonably careful examination, the 
only thing a blueprint used by a carpenter and a blueprint used by an 
electronics technician have in common is white lines on a blue background. 
Thsre is virtually nothing else. Similarly, welding plates on board ship 
has virtually nothing to do with the kind of welding that goes on in an 
electronics production line. There is serious danger of believing we 
have identified general abilities simply because they all have the same 
name. Comparison of stimulus conditions, psychological processes, re- 
sponse patterns, and response modes, frequently reveals that such tasks 
have nothing in common which is Important either for learning or for 
performance. 

The identification of generalizable capabilities, the specification 
of appropriate conditions for learning thsm» tad the description of the 
nature and extent of the generalization to be expected all would be facil- 
itated greatly by an appropriate taxonomy of behavior. At present, we 
lack such classification scheme (but note the useful work of Altman, 1966; 
Gagne, 1965; Melton, 1964) and must tolerate difficulty even in identifying 
tasks which are essentially the same though occurring in a variety of con- 
texts. Without the essential theory, we must resort to empirical deflnl- 

task similarity . This is the problem with which we have contend#* ’ 
and to which you should expect to devote considerable effort. 

Proficiency Measurement 



Our major concern in developing measures of student performance has 
been to ascertain whether a student has mastered the current learning 
task and is ready to go on to a new objective. I believe this is a major 
goal of your plan also. We had a great deal of difficulty in getting 
that kind of measurement. Consider a typical example of our difficulty. 

I was presented with a learning unit which attempted to teach a particular 
machine operation. The test asked the student, via multiple-choice ques- 
tions, to explain how he might do this operation. This test missed the 
point entirely. The proper test would require the student to demonstrate 
that he can perform the operation. Our test writers tended to ask students 
to talk about the performance rather than to ask students to show them. 
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This illustrates the persistent problem of getting representa- 
tiveness in performance measures. Each measure of student achievement 
needs to be representative in at least two ways. First, the measure 
must be an example of the class of behavior which was taught by the 
learning unit. It can't be an example of something else and still be 
testing for the thing you are trying to teach. In the second place, the 
measure must be representative of the class of examples. Consider an 
example In data processing. It may be that you want the student to learn 
to perform all the standard sorting operations with a particular sorter. 

The test you give him certainly has got to be a sorting test. It can't 
be something else. And secondly, It should represent all the sorting 
tasks which he might be called upon to perform. The crucial issue is 
whether, having passed this test, the student has demonstrated that he 
learned what he set out to learn. Getting this kind of representativeness 
is very tough and Is all the more difficult because we lack the behavioral 
taxonomy needed to define representativeness precisely. 

Sequencing Learning 

Our analytic procedure for deriving objectives provided a preliminary 
sequencing of objectives. That is, we started with the most general state- 
ment of what we wanted the student to achieve and we asked, "Now, what are 
the prerequisites for this?" Then we asked this same question of the first 
set of prerequisites, and the second set, and so on until we got to cap- 
abilities we Judged to be within the repertoires of all entering students. 
This procedure was designed to produce an organisation chart or logical 
tree relating each objective to its prerequisite objectives and, thus, to 
define all essential sequences among objectives. But this procedure, which 
requires the same kind of behavioral analysis you propose, turned out to 
be very difficult for many people to follow. The difficulty seems related 
to the need for completeness in stating prerequisite objectives. Whether 
because of problems in analyzing the behaviors involved in an objective, 
or because of a lack of the dogged persistence required for success, many 
people seem unable to develop the sequenced set of objectives which is 
required . 

It should be noted that the problem of selecting the optimum instruc- 
tional sequence for an individual student is not solved completely by our 
analytic procedure, or by any other a priori method. It is possible to 
identify the sequences which must be followed (in cases where A cannot be 
learned until one knows B), but this by no means specifies the sequential 
relations among all objectives. Further, students vary widely in terms 
of their entering capabilities. As a result, sequencing for individuals 
remains a problem for teachers who use the curriculum. They must be pro- 
vided with information as to what sequences are critical, what liberties 
in sequence are available, and on what basis sequences variations can be 
elected . 

A number of practical constraints pose problems in sequencing learning 
units and courses. If your curriculum is to be used by less than an entire 
student body, so that you must share courses, classrooms, facilities or 
staff with the rest of the school, then scheduling and logistics can con- 
strain your learning-sequence plans significantly. Legal requirements also 
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can cause difficulty. Ia some states, for example, it nay be that only 
high school graduates are permitted to enter a course in nursing. In 
such an instance, nursing oust be delayed until the thirteenth and 
fourteenth years. It doesn't Batter that students could learn the Ma- 
terial at tenth grade; they are not permitted to try. All kinds of rules, 
regulations and plans, established by supervisory and accrediting bodies 
to handle circuastances other than those you devise, can constitute limita- 
tions and difficulties until there is appropriate corrective action. I 
don't know to what extent you will have such practical problems in busi- 
ness and office education, but they have been severe in &oiue areas. 

Teacher Training 

Teacher training is a very important aspect of new curriculum de- 
velopment and implementation • I am delighted to see that there is some 
explicit recognition of this matter in the original planning of your pro- 
ject because it probably has been our most difficult problem. There seam 
to be two reasons for this. First, if you are dealing with conscientious 
teachers, they have honed and changed and adjusted over a period of years 
until they think they have considerable evidence that they are doing a 
good Job. In their opinion, they've got something that works. Now, here 
comes something new. The teachers have seen many, many flashes in the pan 
that came and went and didn't amount to anything. They have seen s om e 
which, in their opinion, fell flat on their faces. Now, why should they 
take your brainstorm and use it when they already have something which is 
considered to be pretty good? Well, they can be persuaded, but it is not 
easy. 

The second reason to expect difficulty is this; If I read your pro- 
posal correctly, you will be asking teachers to change their role completely, 
and this is a very difficult thing for them to do. The kind of roie which 
one asks the teacher to assume in "individually prescribed instruction" or 
"individualized instruction" is so different from what they commonly do 
that it takes a great wrench for them to succeed, and many can not. This 
is a problem you must face up to now and plan a solution. Probably I 
should only announce this as a serious problem, because you are more ex- 
pert on how to change people in this respect and on what to do administra- 
tively if you are not able to change them. 

If these observations seem rather pessimistic, please remember that 
my assigned task was to identify significant problems in curriculum de- 
velopment. It would have been an easier task to describe the important 
achievements which cause us to be very optimistic about the improvements 
in education which can result when curricula are developed by analysis 
of performance objectives. 
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APPENDIX B 



SOME VIEWS ON PERFORMANCE OBJECTIVES AND INSTRUCTION 



Albert A. Canfield, Ph.D. 

Vice President-Curriculum, Research and Development 
Oakland Community College 
Blooaifield Hills, Michigan 



A little background of Oakland Community College covering what we 
did, why we did it, and what we found will help you evaluate my recom- 
mendations. 

Oakland County, Michigan (adjoining Detroit) has a population of 
approximately 750,000, an assessed valuation of approximately $2.5 bil- 
lion, and includes both high and low Income areas. It has been estimated 
that we have 30,000 poverty families in the district as well as some of 
the wealthiest familes in Michigan. The college was noted into being in 
June 1964 by a majority of about 200 votes out of over 50,000 cast. 

The college was opened to about 4,200 students on two campuses in 
September 1965. Our enrollment for September 1967 is projected for about 
6,000; approximately 25% of whom will be in career programs — a percentage 
we feel is somewhat too low. 

From the beginning, it seemed that the educational institution should 
key on the learner and his learning experiences. Being an institution 
without a lot of traditiona and practices, it was possible to emphasize 
and consider many points of views. Much of my previous college experience, 
regretfully, was in situations which appeared to be oriented around the 
Regiatrar'a Office, Buainess Office, or the faculty— since I was on the 
faculty, this emphasis seemed appropriate at the time. 

I cannot recall a single meeting or conference during my years of 
employment as a college professor when the concept of improving human 
learning was introduced or considered as the primary topic of discussion. 

The evaluation of instruction in colleges, and perhaps too often in other 
places, ia baaed on appraisal of the qualifications of the teacher and 
not on what the learner gets. There is an unverbalized assumption that 
if the instructor has a lot of information, the Instruction will be good. 

In education the learners are transients, and in any situation where you 
have both transients and people who stay— the transients lose. But even 
though students are transients (some students are in and out so quickly 
they barely get counted). It seemed appropriate to make them our primary 
concern. 

If you were going to start an Instructional program that would key on 
the learner, what would you do? Certainly you would look at the results 
of learning research. Ve did not make a comprehensive, intensive analysis 
of learning theory because we really did not understand it. We can ap- 
preciate so-called principles of learning. Since we are primarily interested 
in humans, we felt that there were principles with which most people seemed 
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to agree; namely, motivation, active participation and feedback or 
reinforcement. There are, of course, other principles but these ele- 
ments of learning were the ones ve chose to emphasize. 

It would also be appropriate to examine instructional research re- 
sults. If instruction is the facilitation of learning--we should examine 
what we have learned about instruction. While learning is characterized 
by complex theories, there does not seem to be an equivalent theory of 
instruction. Instruction Just happens like "Old Man River." The research 
we did find, particularly in higher education, was generally inconclusive. 
McKeachie's chapter in Gage's Handbook of Research on Teaching 1 is illu- 
minating. It reviews 228 articles and studies concerning instruction in 
higher education. These references go as far back as a paper by Ebbinghaus 
in 1913 and includes a few from 1962. There is only one generalization in 
the chapter by McKeachie, and that is, *. • • knowledge of results facil- 
itates learning . . ."2 We have known this since Ebbinghaus, and for 
approximately 50 years have been reconfirming it. 

On the other hand, there are all kinds of research to indicate that: 

T. V. can be good or it can be bad; large classes can be good or they can 
be bad; discussions can be good or they can be bad; color in movies' can be 
good or it can be bad. While people have information that was not in- 
cluded in McKeachie's chapter, the chapter does not necessarily reflect 
everything we know, it is an excellent survey of the field . 

In this regard, I have Just returned from U.C.L.A. where I worked 
with Dr. Arthur M. Cohen (more about him later), and had a chance to re- 
view all the materials in the Junior College Clearing House for research 
on instruction. Only five articles out of something over 850 items in 
their collection seemed specifically instructional research. Most educa- 
tors are disappointed at our lack of research on instruction. There is a 
shortage of hard-nosed interpretable research that can be used for guid- 
ance. Our reaction to the instructional research literature was that you 
could defend almost anything. 

Based on our findings, we decided to examine the traditional instruc- 
tional mode and see if there was a model of instruction that would do the 
Job better. We could not find much evidence to indicate that the conven- 
tional lecture/textbook approach was effective. While it is conventional, 
traditional, easy, ego building, and numerous oth< r things, we did not 
seem to find much information to indicate that it fas effective in pro- 
ducing learning. We considered television and examined the Florida Atlantic 
experience; the Chicago T.V. College, the airborne television experiment 
and felt this approach might create too many problems for us — not because 
the data on television was somewhat questionable, but because we were not 
sure we could manufacture the software to put on the CRT's (picture tubes). 

J-McKeachie, W. J. "Research on Teaching at the College and University 
Levol »" Handbook of Research on Teaching , Edited by M. L. Gage, Rand McNally 
fc Company, Chicago, Illinois, 1963. 

2 Ibid., p. 1155. 
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We looked at compute r-ass is ted instruction (CAI). In early 1965, 

CAI enjoyed considerable recognition. I.B.M. was starting to stimulate 
research on CAI at Pennsylvania State, Florida, Columbia, Michigan, and 
othar places. Both television and CAI were expensive and both required 
a tremendous amount of author time since there were few, if any, programs 
that could be taken off the shelf. It does not seem that computer-assisted 
instruction, operating as a book-printing activity, is going to fly. This 
is a personal conviction based on 1& years experience with it at Oakland 
Community College and in discussion with others. A potential and power- 
ful role for the computer is in the management of learning where it can 
be diagnostic and prescriptive guides. 

Other influences were less rigorous but equally, if not more, power- 
ful. While on a recruiting trip looking for staff, I was asked to talk 
to a college class. After the professor introduced me and I had started 
talking, it was obvious that the students were not listening — they were 
reading the newspaper, nudging one another, looking out the windows and 
reading books. At the tuition rates they were paying, they were paying 
a heavy price, indeed, for the pleasure of attending college. 

On my way back, I stopped by Purdue to see a man called Sam Postleth- 
wit who is the most distinguished teacher and educator I have ever known. 
This man is the Albert Schweitzer of education. You must meet Professor 
Postlethwait , talk to him, and listen to him. He teaches botany in a 
small room in the basement of the biology building. The students are 
engaged in self-instructional activities, milling around learning botany. 
They study at carrels, using tape decks, viewing exhibit tables and re- 
ference materials. The students slip into the pool of knowledge as they 
wish. There are faculty members in the labs to whom the students turn 
for help or discussion. They do not have to try to catch the faculty after 
class. Postlethwait is often in the laboratory with the students, talking 
to them and asking them questions, and helping them appreciate the subject 
matter. The lecture is used very sparingly. 

Even later, we ran across some people who introduced us to the con- 
cept of systems. This is a great time to get into the systems business. 
There is no one person who is acknowledged as the educational systems 
authority. For us, it meant behavioral objectives. I tend to prefer the 
term "performance objectives" because there is a negative reaction in some 
people to the term "behaviorist . " People don't object if you say "per- 
formance." Terminal performance objectives are what people should be able 
to do at the end of a course. 

We adopted the Magerian approach to preparing instructional objectives 
which is very well presented in Mager's book, Preparing Instructional Ob - 
jectives . 1 We also needed carefully designed instructional sequences. We 
wanted to have self-instruction on a self-paced, multi-media basis so stu- 
dents could proceed down one of a number of different paths at their own 



A Mager # Robert F., Preparing Instructional Objectives , Fearon 
Publishers, Palo Alto, California, 1962. 
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rate. I call this the SIMPLE approach — Self-Instructional Multi-Path 
Learning Experience. We also wanted f requent~feedback to the student and 
the staff. These three elements--objectivea, instruction, and feedback 
are what make a closed- loop system. Feedback is used to improve and 
correct media and to assist the student in assessing his progress. That 
is what we set out to do--now what have we found? 

We have found that people ask us for data all the time. When you 
innovate, people say, "Where is your data?" If you had data, you wouldn't 
be innovating. When we get into a corner we say, "Show us your data." 

That serves as quite an equalizer. 

We found that if you know you are going to need research--go at it 
first — get your research designs identified early. Post facto research 
is a very difficult and awkward thing. In any study group (NOELS, for 
example) a research study committee might be appointed right now. This 
committee would develop some ideas about the kinda of reaearch that will 
be done. 

We found that the faculty is the most powerful force in the institu- 
tion. Although we advised people when they came to seek employment' that 
"this is what we are going to do," and naked them if they wanted to do 
that (they had to say, "Yes"), some simply cannot believe there is any 
learning going on when they are not talking. Let me give you an example. 

Here is a student in a carrel and he has a tape deck, some slides, a 
frog, a scalpel, and aome other items like books. We are going to 
have him dissect the frog so he can examine its reproductive system. 
While the student sits there with his scalpel, ilidea, tape deck, and 
is whittling away at the frog — what ia the faculty doing? The faculty 
may be just sitting there. The student is working but the faculty 
member is not likely to feel productively occupied. 

My personal reaction, when I go into a lab, is to go over by a student, 
watch him for a minute, and then if he does not look up and say some- 
thing to me I will interrupt him. I say, "How are you doing?" and he 
says, "Fine." Then, I ask, ,t Whai are you doing?" He replies, "l'm 
cutting up a frog." How, I ask, "Do you understand what you are doing?" 
He answers, "Yes, " I say, "Do you really understand the importance of 
knowing the comparative reproductive systems of animals?" About this 
time, the student catches on. He knows I want to augment and enrich 
his education. So he says, "Ho, I don't believe I do." When he turns 
to face me, I give him a lecturette. 

We have found it is virtually impossible for some faculty to sit 
quietly without either feeling completely non-professional or completely 
unproductive. We wanted them to be available to help students — but at the 
students' request. It is more awkward than we anticipated! 

We spent many thousands of dollars to get 100 plus programmed books. 
They are good books and some are selling quite rapidly. We have found, 
however, that when faculty member A develops the materials for faculty 
member B to use, faculty member B doea not always want to use them. He 
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frequently wants to make or select his own materials. When we tried to 
get everyone to do the same things, with the same material, we got in- 
structional ’'bootlegging." Let me explain. 

We said that every section of the same course in the college would 
have the same objectives, the same examinations, and the same media. 
We found things such as this: One day I saw a faculty member walking 
across the campus with about six youngsters walking single file be- 
hind him--like a mother and young ducks going across a pond. They 
disappeared into a building in a little tight group. I didn't know 
whether this was an exercise in cohesion, a hippy workout, a love-in, 
or what was going on. Later I returned to the same area ana saw the 
faculty member doing the same thing. I learned later that he was 
going into the learning lab where students were trying to study and 
he woula say, "All right, you six people come with me." He would 
then walk over to his office where they would sit and be lectured. 
These sessions would last forty-five minutes and were called "dis- 
cussion sessions." Hopefully, they did not hear about his experiences 
in the war, his first year of marriage, his economic or political 
views, or the other things some teachers often find time to tell, 
their students. After discussion they would be returned to the 
learning laboratory and six other stuaents would get the call. We 
call that "bootlegging" and tried to inhibit it. 

Actually, when we tried to stamp it out, some faculty quit taking 
them outside and continued the sessions in the laboratories. It has been 
very difficult to get today's teachers, with their attitudes and feeLings 
about the individual student, to sit quietly and watch students studying. 

Wo also found that it is extremely difficult to train people to write 
performance objectives, particularly in the soft sciences and English (as 
you heard this last night) and even for some people in technical areas. 
O.ten they write pseudo behavioral objectives. They should be behavioral 
but when examined closely they are not. For example, an objective will 
read something like this: 



"The student will demonstrate his understanding of the theory of magne- 
tism bv correctly answering six of seven items on a true- false test." 

. That sounds behavioral, but it really is not. You can do almost any- 
thing you want under the conditions of that statement. You might have 
seven Ph.D. level exam items, or seven kindergarden grade items. Almost 
any tiling that involves electricity could be used. If you try to convince 
pooDlo that there are bad and good behavioral statements, you ran ret into 
a lot of trouble. If we say that words like "know," "feel.," and "appre- 
ciate" are had and words like "list," "select," and "assemble" are good. 
vou nnv encounter a very reasonable question like: "Are von trying to tell 
- that it is not important whether students appreciate democracy?" Pela- 
' eiv rew pe °P le say: "Yes, we don't think they should appreciate 

de^or.racv at all*" Rather, one says: "Of course, we think st»<der*- s si 0 uld 

aonrociate democracy, hut tne qeustion is how do you tell when thrv do?" 

:'he ":,ood" vs. "bad" dichotomy creates a real conflict between the' behavi or- 
l*- urth h1s °P erant statements and the generalist with his higher concepts. 
. r u mk Oohen at U.C.L.A. has a good idea on this problem and we will talk 
•about u hat later. 
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We found that students often need a Model. They need aosie example 
of what we are talking about. Studenta talk together and they need some 
models of what the program will do for them. One approach la to develop 
a skill hierarchy so that the student starts to develop marketable skills 
from the very beginning. Ideally^ if a student comes to college and leaves 
after two days, he should be a little better vocationally qualified. In 
the business education field, for example, we thought if you could make a 
typist first, then a clerk-typist, you might grow into a stenographer, on 
to an executive secretary and ultimately an office manager. 

We discovered that good self- instructional materials are not avail- 
able and developing them is not as easy as it sounda. Based on Postleth- 
wait's experience, we thought a hard-working and devoted, sincere group of 
people could generate a lot of material. They did, and what a great cadre 
faculty we had! But it takes a lot longer than we thought. If you are 
going to implement an approach similar to ours, you will find that there 
are a lot of textbooks, a lot of Journals, and some filmstrips — and then 
your voice drops. Single concept films, these little 8 mm. continuous 
film cartridges that are getting popular, are quite easy to make. In fact, 
we have found that, at times, it is cheaper to make them than it is to 
find out who made one and see if we can get it. Just the task of keeping 
track of single concept cartridges can keep a half a librarian busy. If 
you want the faculty to develop materials, try to keep the faculty and the 
materials in close contact. The insertion of a media specialist to help 
them can be trouble. Their relationship can develop along these lines. 

A faculty member gets all enthused and goes to the media specialist 
and says, "I want to do this." The media specialist asks, "where is 
your script?" The faculty member states, "i don't have a script but 
I have a Schaefer." The media man says, "Oh, no, no, no. I don't 
mean pen, I mean where is your script, your word for word story — under- 
lined and accented?" When the faculty member says, "i don't have a 
script," he is told that "you can't make a tape without a script." 

The media specialist then spends his time giving the faculty programs 
in script making. So when the faculty comes back with this script, 
the media specialist says, "That's not bad for the first try." And 
then, like a teacher with an English theme, red marks it. The faculty 
member observes, "Boy, this is hard isn't it?" The media specialist 
replies, "Oh, of course, it is hard. You don't think that becoming a 
media specialist is easy, do you?" Or the faculty member comes in to 
make a tape slide presentation and is asked, "Where is your story- 
board?" The same sequence can evolve. 

If you want the faculty to develop materials, I say give them a tape 
deck and a course and say, "Good luck!" Then, stay out of their way. 

Even the crudest effort gives them a basis for what is so desperately 
needed — research . 

Achievement standards are also likely to vary among the staff. Some 
faculty will explain failures by saying, "These are not college caliber 
students. ' With that attitude, the best media, the best objectives and 
the best exams in the world won't help. This "built-in" philosophy will 
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eternally produce a student failure rate. If one asks, "why do you con- 
tinue to wash out half the freshman class?" and you are told, "because 
they are the bottom half" — no matter what calibre comes in or what their 
background training is, there is a selective elite prejudice operating. 

In a visit to an engineering college some years ago, I was told they 
would get all the new students in a big lecture hall and the Dean would 
come out and say, "Now look around you — only SO percent of you will be 
here in two years — ha, ha, ha." It was sort of a promise and they kept 
it. There are faculty who do not think they believe in this, but they 
operationally subscribe to it every time they assign grades by counting 
up from the bottom and giving the bottom 25 percent some kind of un- 
satisfactory or failing grade. We ask, "Do you believe, honestly believe, 
that 25 percent of the students are so ignorant they cannot catch on to 
this material?" And they say, "it isn't that, it's that in comparison to 
the other students, they do average or satisfactory work." 

When one starts developing objectives, there is a tremendous source 
of occupational information available from the United States Employment 
Security Commission and its branches such as Michigan Employment Security 
Commission (MESC), IESC, OESC, and so on. These offices have access to 
volumes of analyses of occupations and Jobs done in conjunction with the 
Dictionary of Occupational Titles. There is a great amount of material 
which can be helpful. Job analyses for years have been using perform- 
ance verbs. By looking through some job analysis one can pick out the 
performance verbs which will assist the faculty. In writing objectives, 
rather than engage in a struggle of what is a behavior and what is not, 
or what is a performance and what is not, I suggest that you go ahead 

and gush out objectives — get rid of them, unblock, flood the area — some 

of them you will later call goals. After you have done that, define the 
behaviors the learners will demonstrate when they have met that goal. 

Early inhibition is negative. If one tries to write an objective and an 
examiner says, "No," it inhibits the flow of further ideas. If someone 

has taken the time to say in his way what the goal of his instructional 

program is, for example, to appreciate democracy — accept it, do not fight 
it. There may be a lot of different ways people might convince you that 
they appreciate democracy. Why not ask, "What are they?" When they start 
to tell you, they are listing possible objectives. Dr. Arthur M. Cohen 
of U.C«L.A.l has suggested this. 

One of the biggest weaknesses I see with conventional Magerian ob- 
jectives is that they are incomplete. Normally, they include a task, the 
conditions under which it will be performed, and the criteria of acceptable 
performance. We have had considerable trouble with this format. After 
2i years I have a positive suggestion. Let me use a simple example to 
illustrate. 



J-Cohen, Arthur M., Defining Instructional Objectives," Systems 
Approaches to Curriculum and Instruction in the Open-Door College . 
Occasional Report from UCLA Junior College Leadership Program, No. 9, 
p. 29-33. 
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Ill writing an objective you say, "The student will," as we do not 
like the insertion of the phrase "be able to." In a physical educa- 
tion course, for example, you night say, "The student will run 100 
yards in street clothes on a level concrete walk with the wind 
velocity less than ten niles per hour in 12 seconds or less." With 
a little study, that could be improved — but off the top of the head, 
that is a behavioral statement. The task is to run 100 yards; the 
street clothes, etc., are conditions, and the 12 seconds or less is 
the criterion. 

For some, when you hand that to the student — that is all! That ends it. 

It is all over! Unfortunately, it is not over — it is Just starting. The 
student quite reasonably, often wants to know, "Why?" He will not always 
ask it, but Just giving him the statement does not settle everything. If 
you put in a rationale for having this objective then you give the student 
some basis for doing it. You tell him why he should pursue this and 
suggest the gratifying kind of feedback he will get when he does this 
task. 



This is not to criticize Mager. His contribution has been funda- 
mental and far-reaching. We can move a bit forward, however, if you put 
in there an answer for the student who says, "Why?" In our physical 
education statement, for example, add the phrase, "in order to avoid the 
police." Now, for some segment of the student body that is motivational. 
They will work to run 100 yards in under 12 seconds. Seriously, you 
might say, "in order that your body will be strong so that you can with- 
stand disease." If we add a phrase, "in order to," or "because of" or 
something of this nature, it gives the learner a reason to learn., In- 
cident ly, objectives are generally in faculty language- -the students talk 
about them and convert them into their own language. Why not put them 
in learner language first? When we write objectives, they are objectives 
for the learner so why not write in the learner's terms and personalize 
them? 



My general reaction is that one needs outside help when training 
the faculty— outside experts if you will. If you have a large group of 
conventional faculty and a thin slice of innovating faculty in the same 
place, the larger group will ask a lot of sticky questions of the innova- 
tors in the lunch room and in the lounge. To even the contest, it helps 
if the innovator can say, "Well, Wilbur said it," or "Postlethwait does 
it," or refer to somebody off the premises. Among other things, the 
innovator needs an outside reference to whom he can divert hostilities. 

In program design, I am a great believer in the principle of "over- 
kill" or, as it is called in education, "overlearning." We are too ready 
to settle for a one-time minimum level performance. We know that over- 
learning is a primary factor in retention. Principles, particularly, 
should be over learned. The engineers overdesign bridges, why shouldn't 
we overdesign instruction? 
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This reminds me of some interesting parallels between instruction 
and the field of medicine. The schools in the United States are like 
hospitals, people come there to get help and then go home. In schools, 
as opposed to hospitals, their stay isn't so long — you are not as con- 
stralnod, and generally leas dramatic physical things happen. The pro- 
cesses of diagnosis, treatment, and evaluation are the same. In school 
we give everybody basically the same treatment everyday and every year. 

It is callecLthe lecture/ textbook approach. If the medics who also 
diagnose, proscribe, treat, and assess, did not do careful assessing, 
we could still be using leaches, burning people with hot irons, and 
bleeding them. In education we do too little assessing. This is what 
I meant earlier when I commented on the need for research. Let me 
elaborate. Consider the medicine man's routine. When attending the 
sick he comes in, throws down a few feathers, does a little dance, and 
calls on the spirits. If the patient does not survive, it is the patient's 
fault or the spirit's fault. The medicine man did the dance right, but 
the spirits were not cooperative or out of the city. 

Can you imagine the lawyer saying, "i have graduated from the finest 
law school, I have been admitted to the bar, I have been practicing for 
fifteen years but, unfortunately, all my clients lose." Or the surgeon 
who says, "i graduated from medical school, I completed my internship, 

I passed all the exams, and I am a member of the College of Surgeons, but 
all my patients die." Or the engineer saying, "i graduated from engineer- 
ing college, I passed the Professional Engineering exam, I am an associate 
fellow in my professional society, but all my bridges fall down." What 
about teachers saying, I am a teacher, I have my degree (s) and earned a 
credential, but my students do not learn." We must emphasize research 
at the instructional level. If we don't routinely assess the effective- 
ness of what we are doing, we will be like the medicine man. 

The design of research, and the collection of data can be only done 
where the action is — between the faculty member and the student. We must 
re-examine not merely our curriculum and our methodologies, but the train- 
ing of teachers to assure a commitment to the consequences of their in- 
struction. 

Well, that is enough conversation from me. Thank you and I hope 
you have some questions. 



QUESTIONS TO DR. CANFIELD 
Q. Do you do any pretesting? 

A. Well, I would have to give an answer something like Dr. Morrison's 
We give an entrance exam at the college called "A.C.T." 

Q. And that's all you have, you don't have any program of individual 
testing or anlysis? 



A. We don't get into individual examination unless there is some 
evidence on the part of the student's performance that a special 
kind of examination is indicated. If it is, we try to stay out 

of that business and get them into the hands of a member of some 

group. We have a fine group of counselors, excellent people. I 
don't think any of them, really, is qualified to utilize depth 
projective techniques. It has always been a little difficult for 
me to figure out who is doing the projecting anyway. We aren't 
against it, we really can't afford it and we don't have, as yet 

anyway, any real indications that we need to do a lot of it. We 

would like to do more pre-testing, but more concerned with the objectives 
of the instructional program than the learning characteristics of 
the student. No doubt, there is a major field for discussion here, 
but we just aren't doing much in the personal counseling areas. 

Q. What kind of procedures are you projecting to follow-up on your 
students leaving the junior college? For example, their grades 
in transfer colleges may not be meaningful for another 20 years. 

A. We are trying, at the moment, at least, to get as specific informa- 
tion as we can on every student who withdraws or who doesn't re- 
enroll. This is no excuse, but as you know, some students just go 
away, they don't bother to stop by and go through the formalities 
of trying to ge their money back and all that. There are some who 
walk in and say, "I quit" or "I've had it," and from those people we 
get pretty good records because we interview them. Sometimes they 
are interviewable and sometimes they are in a hurry and when you say, 

"How come you are leaving?" they say, "l*ve got to go to the Army," 

Now, maybe they do and maybe they don't, but many of them have been 
caught in this trap before where they are given real reasons only to 
be lectured, censured, and humiliated. 

We try to follow up if they go to universities and we try to follow up 
with employees. We are doing it fairly crudely. In the first year of 
the college we had 3,840 students on the fourth Friday. We mailed all 
of them a postcard and asked them to give us some information. How 
many do you think we got back? As I recall, only about 700 cards 
were returned. Then we turned the balance over to the counselors and 
said, "Call them up, will you?" We abandoned the project, because 
we just could not get people to visit, even on the phone. The 
counselors would call their homes and the parents would say, "We 
will have them call you— yes — right away, of course," and nothing! 

We sent postcards, we sent them first class, with stamps you know, 
not junk mail and we didn't get much response. That was the first year. 

With that negative experience, we haven't tried it since. We do 
follow up with the universities and we think we have a good follow-up 
system there. We are not doing much with employers, because we 
just graduated our first class— it was quite small. I will, however, 
give you an example of how data helps. We had 22 students in an 
MDTA career program, and when we graduated 8, ther were a lot of 
comments. "We knew that this caazy thing wouldn't work," "Students 
don't like it," "they don't stay," and the whole routine. When we 
we looked into it, we found that 8 of the fourteen who hadn't grad- 
uated had taken fulltime jobs in the career field before they graduated. 
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We placed 16 of 22, and that isn't bad! Another student entered 
another college fulltime. Another student entered Oakland Co®* 
munity College fulltime and three students dropped for financial 
reasons beyond our control* Fro® looking pretty bad at first, it 

— looks pretty good* But we haven't had the staff, maybe energies, 
or even the wisdom to do this comprehensively* 

Q. I want to follow-up on this first question on testing. You talked 
about the diagnosis of entering student levels and I interpreted 
what you said to mean that in the individual courses, you would have 
some of this. It seemed to me, at least in our own program, many 
of the failures are a result of placing in the wrong place, or in 
the wrong course. They weren't ready for the course and we should 
have done something about it— either in the course, or prior to the 
course. Are you doing anything specifically in that respect? 

A. Nothing much different, really, than most anybody else. The student 
has to see a counselor to enroll, and the counselor has a rule book 
and the rule book says if he hasn't had English I he can't take 
English II and that kind of thing. 

Our long range plan, and we have an impressive computer-based evalua- 
tion system well along, is to pre-test every student in every career 
program and all students in every course. Call that diagnosis. This 
will give individualized learning assignments— call that prescription. 
We will examine student learning frequently — call that evaluation— 
of both the student and our methodologies and media. 
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APPENDIX C 



REACTIONS TO "MOONSBCT" 

DEAN H. WILSON 

Associate Professor of Engineering 
The University of Michigan 



As s person Involved in industrial systems research, I have been 
asked to react to the 'lioonshot" proposal. Candidly, ay reaction is 
one of pessimism. I would like to say a few things about what appears 
in the proposal. 

Throughout the proposal appears the statement: "The systems 

approach will be used in the development of the curriculum for busi- 
ness office education." Now I might ask each one of you, "what is the 
systems approach?" You would say, "Well, it has something to do with 
performance measurement and it has feedback." This is the way it has 
been described, and that is correct . Those parts of a system are neces- 
sary to the systems approach. I might add, however, they ere not suf- 
ficient. You know if you write a proposal to the Federal Government 
these days In Health, Education and Welfare, and put the word "systems 
approach" in it, the probability of funding Increases. But there is more 
behind a system than Just those two words. Lot me give a couple of facts. 
The military, who started using these words back in about 1950, has put 
about 50 million years of work time since the 1950 in systems develop- 
ment for application to military problems. So don't think that you are 
going to earmark Just $25,000 to develop a systems approach to business 
and office education. You are not; system development is long, tough 
work. 




I will not give a specific example of business systems approach. 
You've seen it. Dr. Morrison gave one example last night. He used the 
systems approach in the development of the Quincy vocational curriculum. 
Geary Rummler gave a specific example which includes the systems approach 
in development of the training method for reservation girls at TWA. Let 
me talk about systems a little more abstractly than they have. 

Hero is a curriculum (Dr. Wilson pointed to a system model proposed 
for the talk), there is an output in which we talk about performance 
measures for terminal behavior. I think this is the way both Canfield 
and Morrison spoke of it. Here is an input, "students," which is es- 
sentially the thing that Dr. Balsley keeps questioning, "what are the 
capabilities of the students?" And she keeps asking the question, "How 
do you get a measure?" Tlie process that converts the initial behavior 
into the terminal behavior is called essentially the educational system. 

I guess, from Just reading the planning proposal, it is a sequence of 
elemental construction modules. As I think of it, the modules could be a 
set of a linear sequences of some set theories, some number theories, 
some information, knowledge and techniques for operating machines, how to 
fun a lathe, some work on materials, and so on. These make up what I 

have labeled subsystems within the curriculum modules. 
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Mow, lot me go one stop further. It la necessary that you measure 
the performance at the end, and it la neceaaary to have a feedback loop 
for making changes in the curriculum. But auppoae that we were to pur- 
sue this ayatems approach in developing a curriculum to aee what it will 
look like in five, ten, or fifteen yeara. What would we be doing in 
addition to stating performance characteristics? Well, if what has 
happened in other arena is any indicator, here is what we will be doing: 

We will be able to make a statement like this, "Here ia a learning module 
that has several characteristics. It has subject matter content, it 
has a method of presentation, or a medium, I think you call it, and it 
has a length of time.” If you pursue the systems approach some yearn 
in the future (and by the way, it won't be next year that we will be able 
to make such statements) you'll be able to say. If you change the length 
of time of this particular elementary module from 20 minutes to 15 min- 
utes, the time change will have the following specific effect on the 
terminal behavior, a scale against the length of time of learning module 
No. 7 in the curriculum and how it affects the terminal performance. 

These kinds of data about the relationships of each one of the learning 
modules to overall terminal performance are what you will have if you 
pursue a systems approach. You will also have some understanding about 
the interactions between two modules or pairs of them. I think most of 
the curriculums that are developed are done so on the theory that the 
modules are independent, or so Mr. Rusmler tells me. I suspect, however, 
that you will find that they aren't independent, especially with respect 
to all students. There is some interaction between modules, and if we 
pursue the systems approach in the development of curriculum, we will 
understand these interactions. 

A word about the differences between the analog and mathematical 
model. A medical researcher using a guinea pig says, *Here is a system 
that is similar to the human being. In my experiments on the guinea pig, 
as I observe the results in the model and insofar as the model is like 
the human being, the conclusions are valid for human beings." The model 
could be either analog or mathematical, and I don't know which would be 
most appropriate for education. I am not saying mathematical. But here 
is the next point that I wanted to make. This is a curriculum (points 
to illustration) following a systems approach. The approach allows us, 
say in five years, to be able to compare alternatives. We will be able 
to compare two curriculums and determine essentially which comes closer 
to meeting the desired terminal behavior that someone has agreed is de- 
sired from an educational program. In other words, the ability to com- 
pare alternative proposed curriculums is one of the capabilities that we 
pursue systems analysis. 

For example, the air force is very good now at making comparisons be 
tween two proposed fighter aircraft they are using in Viet Nam. They are 
very good at making these comparisons, by the way. Fifty million dollars 
a year for fifteen years gave them the capability, don't forget, but they 
are excellent at it. I might say in this context, if we were to ask, 
'Suppose we take a particular fighter aircraft and put it in action in 
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Viet Nam. Suppose, further, that we give it this kind of a mission. 

What is the probability of its survival?" That, by the way, is cal* 
culated in the air force today. You send the fighter over today; you 
do the mission tomorrow; and the accuracy of that prediction is within 
1 percent. 

How has this high degree of accuracy been attained? In the States, 
the air force has a model of the process of the environment in Viet Nam, 
the air force and its performance capabilities. The air force has de- 
veloped the model of the process by collecting all data on the process. 

In education, too, all data of the system process will have to be collected. 
Why? In order to build a model of the educational process so that ex- 
periments can be run on the model without going out to the real class- 
room and students for every new idea. 

Thus, we must move further in developing models of the educational 
process. We must experiment on the model in exactly the way a doctor 
uses a guinea pig or the monkey when he runs experiments on the circula- 
tory system. Then, we can use results of the experiments on models to 
predict that when making given changes in the curriculum, the changes 
made will produce such changes in the performance capabilities of student 
outputs. So the development of the model for business and office educa- 
tion process is going to be a long, long task. Based on experience in 
other areas, I don't think that you will have one that will be very good 
for at least ten years. You are not going to say, "Well, let's develop 
the system tomorrow." And it is not going to be done for Just a few 
million dollars. It is a tough problem. 

We can certainly learn a lot from other people who have developed 
systems in other areas. I think I would look into the educational de- 
velopment programs that the military has. I'd look at the military train- 
ing program and the models that have been developed from military train- 
ing programs. How do they train technicians? How do they train super- 
visors for radar installations? These jobs are really complex but not 
different from those in the business offices. Thus, because the military 
services have had lots of money to develop systems, I'd attempt to go 
there first. 

As you can see, my reaction to the words "systems approach" in the 
"Moonshot" planning and feasibility study is that this group probably 
doesn't really appreciate the difficulty of developing a system. 

Now, there are a couple of other points that I want to make. The 
next point is that an educational process for producing students who can 
get jobs and maintain jobs in the business offices exists in an environ- 
ment. I think you talked about this environment, and it is essentially 
the business office. There is an environment which is exclusive of the 
office environment; it is not independent of what is required. Essentially, 
it is the student population from which you pick students to put through 
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the curriculum. For example, changes in the characteristics of the 
students will require that changes be made in the curriculum. It may 
be, for example, that your curriculum may include a module on teaching 
sets in arithmetic. Due to changes made in the fifth grade, though, 
all students may be learning sets. So you take instruction on sets 
out of the curriculum and don't repeat it; or at least you may want to 
modify it. So that is how the environment in which your curriculum haa 
to operate affects the curriculum. 

I am still reacting to your proposal and ideas I have received from 
it. In considering the changes in the business office, the technology 
that is used there, the method of operating, the management skills that 
are used in business office; these changes are certainly going to feed 
back into the system and affect the curriculum. We might think of theae 
office requirements in two categories: (1) In the proposal you call one 

category the "job entry skills." I define job entry skills as those 
skills that primarily have to do with manual dexterity. These are es- 
sentially the implementing or the technique of performing a function in 
a business office. (2) The other kind of skills that are involved, I 
have labeled "adaptability skills." Those are the kind of skills that 
give the student the ability to maintain a job, to retain a job under 
conditions of changing technology and development in the way a business 
office operates. So one feedback loop is really the changes in the re- 
quirements for job entry. 

It may be and it look£i like it will be a very few years before some 
of the typing may be obsolete as a job entry skill. I think maybe that 
is a poor example. Let me give you one that is really going to be ob- 
solete as a job entry skill, although it is a very high priority on the 
list of needs now. That is keypunching. Keypunching is probably one of 
the most popular types of things that can help students get a job today, 
and within five years it will be obsolete. So changes in job entry re- 
quirements for job entry skills will affect your curriculum. 

Another feedback loop from the business office will be changes that 
take place in those skills that are required for a person to maintain a 
job or to adapt to the changing business office. These changing skills 
are a little fuzzy and I am going to spend some time this afternoon, 
hopefully with my panel, talking about them. I have gone through your 
proposal very carefully and listed everyone of the characteristics that 
you have said a student must have if he is going to be adaptable to the 
changing business world. I want to ask how you are going to build a cur- 
riculum that produces adaptability behavior in students. 

Let us summarize the way the curriculum can change: one is that 

the student population changes; one is that the business office charac- 
teristics change either in the job entry skills required or the adapt- 
ability skills needed. What do we measure? We have to measure, es- 
sentially, the percentage of people who, when they come out of this pro- 
gram, get their first jobs and those who don't get one . These two 
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measures together help you decide whst changes to make in the curriculum. 
How many people get their first Job in a business office but are un- 
employed a year later? We have to learn how to measure those that can't 
retain a Job; we must keep records on retention because that is an im- 
portant feedback loop for helping us decide how to make curriculum changes. 
I have listed four types of changes that one has to acoount for in the 
development of the business office curriculum and what has to be measured 
and fed back from the business office in order to develop curriculum 
that produces a student that can get his first Job and maintain his Job. 

I guess if I had read John Gardner's book before I developed this chart, 

I would have recognized the self correction of business office education 
curriculums. I think I would call this "self renewal," because that is 
what I have tried to take into account. 

How do we keep adjusting and changing the curriculum? In engineers' 
Jargon, we say that the environment is not stationary and we must adapt 
to changes: we must measure what the system is doing and adapt to changes. 

The military has accomplished this. They keep testing the environ- 
ment in which their military systems operate and feed these environmental 
tests into their model of operations, thus deciding how to adapt the mili- 
tary systems to maximize their performance. 
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k 4w J ? St B Side coa,ient » tho Michlgan-Ohio Regional Education Laboratory 
uilt its last proposal to the U. S. Office of Education on the very con- 
cept of self-renewal. Well, self-renewal appears in your first proposal, 
too. It isn t developed and doesn't really go far enough. The words 
appear, but I believe that for the continuing project I would certainly 
want to develop the idea further. 9 
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Noras ON "MOONSHOT" - PROBLEMS AND SUGGESTIONS 
F. Carpenter, The University of Michigan 



1 • C hoaaiqg gaong alternatives for developing the system . It may 
be profitable to consider different approaches that can be used for 
making the product. The final choice may be important because it re- 
lates to uses of available resources (budget, etc.). The three main 
approaches that occur to me are: 

A. The de novo approach which means that minimum concern will be 
paid to the strong aspects of current programs. Instead of borrowing 
what may already be usable in existing curricula, we plan to start from 
scratch and build an entirely new system. Adoption of this approach im- 
plies that the project will have plenty of money, time, and available 
manpower to do the Job. If it appears that any of those resources may 
be somewhat limited, choice of another alternative seems advisable. 

B. A second approach would be to select several existing systems 
that have high success indices, as measured by acceptance of the grad- 
uates on the vocational level. This approach offers freedom to enter, 
study, and modify these systems which may reduce time and effort to com- 
plete the project in comparison to the first approach. A second advan- 
tage may be found in rapid assessment of the adaptability of new compon- 
ents within existing systems. A third advantage relates to market- 
ability. By demonstrating that the program is a definite improvement 
over current programs rated as exemplary, the reaction is likely to be 
quite favorable by prospective users. 

This alternative would involve an intensive initial survey of busi- 
ness education programs, and then rating them as carefully as possible in 
order to choose a sample of exemplary programs for the purpose of im- 
proving them via the systems approach. (Some remarks by Morrison can be 
Interpreted to support this alternative.) 

,C. The third alternative may be called the "piecemeal demonstration 
plan which attempts to examine the comparable values of the new compon- 
ents with their existing counterparts. Marketing is a main concern here 
because the aim would be to provide customers with a choice of buying 
the whole system or any of its units, any single package being made to 
work in the traditional system. Various curriculum packages would be 
developed simultaneously in different schools and tested centrally to 
fit into the total system. The idea, an it appears on paper, seems to 
have certain economic advantagea. Yet, it presents a challenge to the 
system makers because it means that not only the entire system must have 
general utility, but that each of its nub-systems or components can be 
adapted profitably to existing systems. 
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My own suggestion is to choose the second approach because of 
promised economy and hopefully its demonstrated superiority over pro 
grams now considered as outstanding. 



II. The problem of •too much success .” Is it possible that success 
of the project may actually create problems on the vocational level? 

This appears, on the surface, as a most inappropriate question, but let 
us consider a probable outcome. Assuming that all graduates of the new 
system will acquire more than office skills taught in traditional pro- 
grams, that they will learn the nature of the business enterprise, its 
purposes, functions, and relationships, what psychological climate is 
likely to result between the brighter graduates of the new system and 
their on-the-job superiors, who are not used to such erudition 

in the subordinates? It seems likely that the modestly bright new grad- 
uates will have a better grasp of the total picture than most of their 
clerical bosses. They may appear to be upstarts without even trying. Un- 
less this probable result is anticipated and provisions made in the system 
o cope with it, the success index of the program may be less than it 
actually merits, since this is not an unlikely outcome, it implies that 
appropriate provisions be made in the form of a coordinating sub-system 
between executives in the hiring offices and the training system. It 
may even require the introduction of an orientation program that will help 
the hiring offices best utilize the talents of the new graduates which 
mean that the degree of utility of the product of the new system may de- 
pend upon some education introduced at the vocational level (in hiring 

offices), particularly for clerical bosses from the old school who are 
not themselves executives. 



The amition of Moonshot is to create a new population of abilities 
in the behavioral repertories of clerical workers. It seems that the new 
additives will overlap with the repertory set now held by junior execu- 
t ves. Theoretically, this should provide for better communication be- 
tween executives and non-executives. But we should be prepared for an- 
other influence of the program that has apparently not yet been antici- 
pated. Namely, that the new curriculum will likely raise the aspiration 
level of the brighter students, who, because of their increased compre- 
hension of the business enterprise, may not want to terminate their school- 
ing at the high school level. The program may inspire them to attend 
Junior colleges, community colleges, and special business schools in uni- 
versities. This likely outcome would require the modification of the 
success index of the new system to include the percentage of graduates 
that elect higher education. 



HI* Problems of adaptation training for student s . 

A. The possible disparity between the pattern of habits best suited 
to adjustment to the training system and to the vocational environment 
presents one problem of adaptation training for students. Successful ad- 
justment to school does not guarantee successful adjustment to the world 
of gainful employment. Theoretically, one advantage of the systems ap- 
proach in curriculum building is that the self -correcting devices may be 
used to bring the demands of training nearer to the demands of post- 
training environments. One of the main psychological problems involved 



in this consideration is that successful adjustment, in large part, 
neans learning how to cope with streas. Consequently, it seems logical 
for the planners of Moonshot to Bake special preparations for dealing 
with this problea. Selection of training procedures to iaprove flex* 
ibility of adaptation to the noraal range of office conditions in the 
field should probably include: 

1* Various siaulation techniques such as mock-ups of offices 
having different kinds and degrees of stress. The idea of training 
for adjustaent to streas appears quite profitable as suggested by in- 
spection of military training prograas. (Exaaple: Extra deaands of 

bosses irrelevant to job skills.) 

2. Siaulation by gaae techniques. 

B. A second problem involving adaptation training is iaplied by 

this question: How much information on individual students should be 

available prior to entering the program? If flexibility is one of the 
main concerns of the program, that is, the intent to train the student 
to have desirable flexibility in certain situations, it seems likely 
that some kinds of rigidity of students aay prove uneconomical to bver- 
come by the system. If so, then it seems advisable to explore the 
available field of measures of various forms of rigidity or to ab- 
stract the symptoms empirically. Although Moonshot apparently cannot 
indulge la the luxury of screening students to the point of pidtiigonly 
the cream of the crop, it seems wise to identify, if possible, those 
characteristics (habits) in students that promise to resist change in 
ways that are desired, while only an empirical examination of the pro- 
blem may solve it, the idea is that information about likely parameters 
of individuals concerning flexibility may prove quite useful. A prac- 
tical residual value could occur in adopting this suggestion by pro- 
viding valuable information to school counselors. 

C. A third problem in this area, closely related to the preceding 
one, is the task of finding other pretraining correlates with patterns 
of success under various conditions of training. I think that this con- 
sideration is closely related to the main criterion for judging the 
success of any educational system. The value of innovations in modern 
schools must be Judged, in keeping with the temper of the times, by the 
extent that certain changes demonstrate successful accommodations to 
important individual differences related to learning effectiveness. I 
have seen a number of new creations of educational technology that are 
hardly more than expensive toys that serve mainly to disenchant students, 
teachers, administrators, and the public. It seems important to have 
sufficient (pretraining) information on students so that cluster lana- 
lyses can be accomplished that will eventually provide predictive data 

on optimal paths 



^■See Professor M. Clemens Johnson for further details. 



of the teaching- learning aaze beat suited to particular kinds of students. 
Opinions differ radically on the amount and kind of pretraining informa- 
tion needed to achieve that end. Some say that only a minimum of entry 
behavior is required to judge where the student should begin the program, 
and that the student will provide all the other necessary information by 
the way he behaves during training. But others have disputed that posi- 
tion. Their work (for example, McKeachie's) indicates that decisions 
on how to alter the psychological environment seems to call for somewhat 
extensive information on the student himself. The promise seems to be 
that considerable knowledge of the student concerning his habits, values, 
inclinations, interests, and the like may improve the efficiency of 
decision-making in altering the system by reducing some of the cut-and- 
try procedures that would otherwise be used. Assuming that this is an 
important consideration, the practical suggestion boils down to this: 
somewhat extensive descriptive data should be gathered on students of 
both cognitive and non-cognitive kind and the data then be subjected to 
cluster analysis to determine patterns that can be conveniently related 
to success and failure at various stages in the program. The kind of 
preparation and practice may prove to obviate some of the costly trial 
and error procedures which are likely to occur in the absence of such 
information. (Example: program vs. aptitude data) 

IV. The problem of teacher educator — regular teacher subsystem . 

A. The first problem that is likely to occur in this subsystem is 
the lack of rapport between the teacher educator and the regular teacher. 
Because this system is designed to place the teacher in the position of 
decision-maker in determining the selection of alternatives open to the 
student, the teacher who is learning to make those decisions must be 
kindly predisposed toward the new system. If the situation comes to be 
aversive to him, he could wreck the whole system. Consequently, some in- 
service preparation in the way of carefully planned orientation procedures 
should be considered. Careful selection of the teacher educator also 
seems important to avoid a personality clash between him and the regular 
teacher. 

B. A related problem is the task of getting information on the 
regular teacher that can be used to estimate his success in adjusting 

to the new system. Explorations in this area are recommended with perhaps 
some consideration given to getting certain noncognitive measures that 
logically apply. 

V. The need for a placement bureau . The fifth problem is indicated 
by the absence in the systems diagram of a needed component, namely, a 
placement bureau which could be used to gather information on both grad- 
uates and characteristics of the various vocational environments among 
which they may choose. The problem of matching applicants with offices 
may be needed to insure a high success index. The range of working con- 
ditions and psychological atmospheres among offices is no doubt somewhat 
broad. When the market favors the applicant, he should be given help in 
making a choice by providing him with more than the usual kind of data. 
Psychological matching requires information on the office as well as on 
the applicant . 
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VI. Problems of assessing student per forms nee . Since Moonshot seems 
to parallel the Quincy, Massachusetts project rather closely in its de- 
mands for evaluation of achievement, I would like to raise some extra 
considerations that are not indicated in the Quincy proposal. 

One consideration is the task of choosing and maintaining useful 
cumulative records of performance during training, it seems advisable, 
whenever possible, that continuing records of learning be established in 
the modules so that relationships can be inferred between how the student 
behaves and changes in the instructional stimuli. That end is fairly 
well accomplished when proper management is applied to programmed materials. 
But the challenge of invention seems to arise because techniques do not 
seem to be available for use in all situations that seem both desirable 
and possible. The programmed instructional paradigm, however, may help 
serve as a model that can be approximated through inventive measures. The 
important value of good cumulative records of performance during learning 
is that a fine-grained analysis is made possible that can serve as the 
basis for making useful changes in the instructional situation. A guiding 
principle, may be to maximize the establishment of cumulative records 
of performance with associated changes in conditions; and when either- 
or choices must be made, to choose in favor of the cumulative record as 
perhaps the best criterion for assessing the effect of modules on individ- 
uals. Another suggestion along this line is to explore the utility of 
the technique of interaction analysis (Flanders) and examine some of its 
possible modifications. 



EPILOGUE 

After listening to other speakers on the program particularly 
Morrison and Canfield, I think that the building of effective instruc- 
tional systems must involve a strong emphasis on the teacher variable. 

As Canfield indicated, the psychology of the teacher cannot be ignored. 
Unless successful effort is introduced to modify the habits and inclinations 
of teachers that are incompatible with the system, then the teacher is 
bound to behave to create costly noise in the system. Canfield's re- 
ference to Professor Postlethwaite 's outstanding success in raising the 
achievement level of his students with only modest facilities, including 
some crude equipment, is a strong testimony to the strength of the teacher 
variable. Since Moonshot defines the teacher's role as a central decision 
making agent, it becomes vital to have teachers properly trained to operate 
in the system. I am now inclined to think that the teacher variable is so 
important that, in general, the building of educational systems should 

with teacher training; that is, by first making a system devoted to 
the training of teachers in the understanding and operations of instruc- 
tional systems. If only a cursory treatment is given to this important 
problem, most of the effort is likely to be wasted. 

A second important modification may be a tough one to make at this 
point because so much effort has already gone into the development of the 
project along certain lin^s. The suggested modification is initiated by 
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the question: What should be the nature of the first proposal that will 

maximize the probability that it will be funded? The main handicap so 
far is that so much work must be done in such a short time. Consequently, 
it seems likely, despite the apparent importance of the project, that 
the time pressure may prove too much of a handicap unless some careful 
thinking is done to obviate precipitous decisions. The first atep is to 
consider the great complexity of the project and the amount of time, 
effort, and special counsel needed to develop it properly. I think that 
the project’s complexity is obvious to all concerned, but the desire 
to produce a fundable proposal in such a short time could easily result 
in the omission of important details from the standpoint of the reviewers. 
More time seems to be needed to write the most defensible proposal. How- 
ever, because the time factor is such a serious restriction, careful 
thought* must be given to choice of alternatives open for making a pro- 
posal that is most likely to get funded. 

My suggestion, under these circumstances, is to aim for a pilot in- 
vestigation as the first phase so that decisions concerning later pro- 
cedures can be made on more than simply educated guesses. Yet, the 
pilot study must be seen as an integral part of the whole package. My 
suggestion is that the pilot study be largely a fact finding operation 
to determine the economy of the various approaches. Before the group 
is finally committed to an approach, it should be fairly clear that suf- 
ficient groundwork has been done to show that the chosen approach will 
promise to be the most effective and economical among the alternatives. 
Because of the size of Moonshot , it is reasonable to initiate the study 
by a careful examination of the possible routes that lead to the goal, 
and to be able to show that the chosen route is the most feasible among 
those that are available. Consequently, I suggest that the initial phase 
be addressed to the examination of current curricula in business educa- 
tion for the purpose of identifying the most effective and usable com- 
ponents that now exist. Secondly, an analysis be made of those compon- 
ents to be accomplished to show their relevant and weak points in re- 
lation to the demands at the vocational level. Also, a study needs to 
be made to describe the changing demands in the office world so that the 
new curriculum can mesh with those demands. I would suggest that a later 
phase be used to develop the details of the systems approach after proper 
analysis has been made of the necessary facts that describe the status 
quo adequately. The suggested sequence of tasks in the initial proposal 
should probably show the following order: 

A. A study of current demands of the business world to estimate 
the rate of change in those demands and to clarify the lag between them 
and current curricula. 

B. A survey of exemplary programs that already exist. Because the 
de novo approach appears to be unnecessarily costly, it seems wiser to 
know in detail how adequate the best current systems are and attempt to 
improve them rather than starting from scratch. (Morrison has already 

indicated the unfeasibility of making all materials by starting from 
scratch.) 
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C. A survey of the Management techniques of exemplary programs to 
determine the ralative effect of management vs. content. I have ob- 
served directly hov expensive systems can fail, even when they have good 
equipment and materials, by inadequate teachers and management. The 
importance of this part of the initial survey is to get some estimate of 
how much the Moons hot system should address itself to development of 
managerial techniques of the learning environment and how much it should 
emphasize alternation in content. The latter concern (modifying content) 
is best judged as the result of "a" above. 

D. At the conclusion of Phase I, which would Include the above 
steps, the builders of the proposal should supply the design of the 
Moonshot system. 

I am inclined to think that the adoption of the above plan or of 
something similar will not only reflect that the architects of Moonshot 
have considered the vastness of the task and have not been stampeded into 
making precipitous judgments largely for the purpose of gaining a hugh 
grant, but that they have adopted an approach that reflects that their 
primary concern is the making of a significant contribution to the- field. 

The main principle that I've tried to reflect in this counsel is: 
the success in completing complex projects is best insured by breaking 
the large problems down into researchable parts. If the Moonshot pro- 
posal fails to convince the reviewers that the proposed action accords 
with the principle, its future will be rather dark. 
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APPENDIX E 



CONSTRAINTS IN DESIGNING NOONSHOT 



HARRY HUFF1GAN 

Program Specialist in Business and Office Education 
Center for Vocational and Technical Education 
The Ohio State University 



I have eleven points to make about constraints that must be observed 
in designing H oonshot . They may not all be valid, but I hope that you 
will help me revise and add to them: 

1. One constraint on the research project is that we must be con- 
cerned with business and office education, vocational education, and I 
think most everybody accepts that • It is very important that the members 
of the business education study committee be sure to discuss this with 
people over the nation because some of these people may thlitk we are too 
narrow , Our limitation to vocational education was caused by practical 
concerns, i.e., funds are available for research and demonstration here 
bat not so certainly in the general business or economic fields# We 
ruled out distributive education because most of the initiators are 
specialists in business office education . 

2. The second constraint is that we are emphasizing, as well as the 
manipulative skills in our field, the thinking part of office education. 

In other words, we are trying to conceptualize our field no something 
that makes a contribution, for example, to business and office Information 
processing. We are trying to define our field as a major part of the 
American economy, something that maybe we are giving lip service to now. 

I hope that we can get thinking into our research and development pro- 
ject; I hope that we continue to recognize that we are dealing with an 
' area of information processing in this particular project. 

3. The third constraint is also one that people in the field will 

want to know about: Oiar immediate target in Phase i is the high school, 

9th through 12th grades; other levels will be considered later. The ration- 
ale behind focusiiag on the high school (grades 9-12 first) is that this 
area is where the major segment of our business and office education stu- 
dent population is located at this moment in time. 

4. The fourth constraint recognizes that roughly 75 percent of a 
student's time will be outside of our field. This constraint means that 
when we talk about educating the student, we recognize that there are 
other aspects of the educational program that will have* Impact on stu- 
dents. In other words, we want to turn out a citizen, a self-actuating 
person, from our schools. Business and office education doesn't and 
can't do it all. We are going to recognize that the other part of the 
secondary school program makes a contribution. 
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5. The fifth constraint relates to instruction. Materials must be 
flexible to fit the present system of schools. Yet materials must be 
flexible enough to adapt to changes in schools. In other words, we assum 
that we are going to develop materials for present school systems but 
change as school systems change. 



6. "Moonshot" will be concerned with all boys and girls that are 
high school level and particularly those of average and lower than average 
aptitudes and achievement. So this constraint recognises that we will 

be dealing with all kinds of boys and girls, but we must be concerned 
particularly with the average and less than average. 

7. We must recognize and temporarily screen out boys and girls who 
aren’t ready for the program. If the reading level is at the third grade 
level and the arithmetic fourth grade, perhaps we don't have any business 
working with them. Perhaps for their sake, these poor readers and com- 
puters should develop these skills to a point that they can profitably 
follow our programs. 

8. Another contraint is the classroom teacher and his concept of 
himself. In the United States we have 60,000 or more business and office 
teachers, many of whom are teaching subjects we are proposing to change. 
We must be concerned that we develop marketable materials, materials not 
too out-of-line with concepts of present teachers. 



9. Contraint nine relates to the current status of business teacher 
education. It certainly doesn't apply to the people here, but many 
teacher education programs in our field are insulated from the firing 
line. The teacher on the firing line and the city supervisor on the 
firing line get great pressures to make changes, and I am making an 
assumption that teacher education people are back a step from that. They 
don't feel the pressure as much. We must recognize the current level 
of teacher education as a constraint but recognize it as one we hope can 

be moved. 



10. Number ten is a constraint due to concepts held by school admin- 
istrators. It is their concept of what our field is. I am sure we have 
B job to educate the* but I am saying that their concepts are a constraint 



11. Number eleven is a constraint due to the concepts held by school 
counselors on what our field is and what we can do about it. I am sure 
that there is an enormous constraint on us because no matter hew much we 
talk, how much we tell them that business and industry can use all levels 
of abilities, they seemingly don’t listen. So, I think counselors con- 
cepts are a constraint . 



Question: All these are constraints within our profession and nothing 

from business. Now, do you feel there are no constraints so far as the 
business world is concerned? 



Answer: That is a good point. I think that we are limited fre- 

quently. When we make surveys of what the duties and activities are, I 
think we get a stunted picture from many business people, a very stunted 
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picture of what goes on. If you do anything like using a ter* ’’systems 
analysis,” you find that they don't really know what goes on in their 
businesses. They don’t know what goes on because they haven't made an 
analysis. So, when businessmen tell us what they require, they are 
basing their requirements on a very limited picture. So I think it is 
a good point. 

Question: Are you suggesting then, that we add a twelfth con- 

straint: — a constraint of business demands? For example, typewriting? 

Answer : Yes • 



© 
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APPENDIX F 



ESTABLISHING PERFORMANCE OBJECTIVES— A CASE STUDY 

GEARY RUMMLER 

Director, Center for Programed Instruction for Business 

The University of Michigan 

Chicago, July 30-31, 1967 



I'm concerned with the problem of establishing terminal behaviors 
or performance objectives. I want to explain to you what we do in the 
way of analysis that leads to the statement of objectives. Later, would 
you react on how you think our system will work, will not work, and 
what really has to be done to implement such a course in Moonshot ? 

I'm going to describe to you very briefly how we attack a problem. 

I'm presenting to you, not relating directly to Moonshot, how our group 
at the Center for Programmed Learning attacks performance problems. 

Briefly, here's a case history of how we look at a performance prob- 
lem. Specifically, TWA contacted us earlier this year about a four-week 
course they use at Kennedy International Airport for reservation agents. 
Reservation agents are the girl? that sit all day, taking calls from ob- 
noxious travelers, trying to get them ©£ the right flights, and answering 
all their questions. When you call an airline, you will probably be talk- 
ing to one of the eighty reservation agents in your area, sitting in a 
long row similar to an assembly hall; just sitting there taking calls all 
day. The job is a very complex one, requiring four weeks of basic train- 
ing. TWA was Interested in reducing basic training time considerably be- 
cause the cost to them is in the neighborhood of half a million dollars a 
year in per diem. Thus, if they could cut the training from four weeks 
to three weeks they would save $125 thousand in per diem costs. You who 
are connected with Moonshot aren't in the kind of situation where that 
kind of dollar value can be placed on students. Our feeling is, however, 
that the student unconsciously has a dollar value on his time as he's 
sitting through a course of instruction* 

In looking at our task, we set out not necessarily to change the 
course or the curriculum. Our first step is to look at the job and see 
what is Involved. Forgetting about training and how we are going to 
propose doing the task, we look at what people actually do. We visited 
a number of different reservation offices around the coutMtxy, talked to 
and watched reservation agents (two of our people learned to be reservation 
agents), sat at the phones, completed flow charts of every major decision 
a reservation agent has to make. Seven or eight tariff manuals, we found, 
had to be referred to with seemingly Infinite number of combinations of 
schedules that passengers can make. 

After we discovered what the job was, we talked to the management 
and talked t <r the training people of the organization. More importantly, 

we tried to find out how the job really should be done, tried to identify 
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the 15 or 20 percent of the tasks that workers do incorrectly or that 
management wished workers would do differently. Thus, we made an effort 
to find out what the Job should be, what it is now, and then tried to 
figure out what the tradeoffs were to be. The reservation agents can do 
several things for the company after they have gained experience. They 
can become what are called "Agents," a kind of super-agent, who walks 
around between the rows and handles difficult problems; or they can be- 
come a supervisor. Again, we had to find out what the line of pro- 
gression was, what skills were involved, decisions involved, and what 
the most common types of deficiencies of people were. 

What comes out of this kind of analysis with our doing the Job, 
observing all people, talking to supervisors about what are the three 
most common deficiencies is a series of documents. One is a complete 
chart with the major decisions a person performs on that Job and what 
they do. We end up with a list of terminal specifications or performance 
objectives for every subpart of the Job, which basically says, "Given 
a telephone call of this type, the agent will be able to handle this 
kind of call correctly within a certain number of minutes." We f ve also 
come up with a third thing. First we have what the Job is; second, 
the terminal specs, and the third thing we’ve come up with is what we 
call "consequence of performance analysis." For example, management 
say, "He should be doing these things," when people are doing this. (We 
tend to err in the direction of the person who is doing the Job) • If 
he’s doing it that way, there is a very good reason why he's doing it 
in that particular situation or environment. For example, company policy 
says that when a customer calls, the TWA reservation agent should try to 
sell that customer a tour of the city. If one is going to Chicago, the 
agent should say, "Did you know that when you're in Chicago you can get 
this kind of tour?" That's what company poMcy says. 

We talked to the girls and they said, "Do you see that big board up 
in front that looks like a football scoreboard with those little red 
lights? Those blinking red lights are the number of calls that are on 
hold that no one is taking care of. Then, there is my performance evalua- 
tion sheet containing the number of calls I handle per hour, per day, 
and also the percent of sales I make c Now you don't think I'm going to 
stay on this phone and talk an extra two minutes to someone to sell him 
a tour which isn’t credited to my record, do you?" 

Thus, given the desired behavior we want, then what happens if the 
worker does it or not in terms of dollars, supervision, or any other 
variable? In other words, we find out why the system is the way it is 8 
Then we talk to management saying, "Our advice, sir, * s change your 
policy and you'll get more out of the real world." That's a third kind 
of major document that comes out of this analysis. 

Thus, in our procedure, we have concentrated on performance, we've 
identified deficiencies, and we've tried to locate whether the deficiency 
can be corrected by training or whether, in fact, there's a whole pro- 
blem of the environmental control. 
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Nov cornea the more difficult job, i.e., how to design • course to 
attain the specs. The course that we've designed for TWA cut the train- 
ing to less than two weeks and did that by a number of decisions by our 
staff involving some consideration of learning principles. Let me 
describe the kind of decisions that are made at this point: One of the 
tasks that a trainee has to do at TWA when she takes an order is to fill 
out the sales order form and put it on a conveyor bfelt going to the 
bookkeeping department. To teach this task required approximately eight 
hours in the present system at about the third week of training. Our 
people took a look at the situation and decided that filling the sales 
order form was a terminal behavior that the sales agent goes through at 
the end of every task. That is, almost every task that she does as she 
makes a sale, ends in doing this kind of behavior. Why not teach it 
very early, say the second day? We don't have to teach it in great de- 
tail because we will now use the sales form as the major form of re- 
sponse for all training for the rest of the week. As we teach the use 
of the telephone, we teach trainees to use the manuals and we will always 
say, "Now that you've done this task, fill out a sales order." So it 
takes something less than an hour to teach the rudiments of a sales order 
early in the course design and it is reviewed through continuous use. 

That was one decision which changed an eight-hour task to a one-hour task. 

Another example is that we want to emphasize selling because reserva- 
tion agents are really sales people. We should start early in the four 
weeks teaching them how to sell and the importance of selling. We ended 
up making selling the last three days in this design because much selling 
behavior is merely talking on the phone, and these are behaviors that 
people already have. Looking through four different manuals and .calculat- 
ing rate structures is a very difficult and very foreign task to agents, 
so the decision was made to teach the manuals early without introducing 
the phone which is an interf erring behavior. Bring this skill up to 
strength and finally get agents on the telephone where again they review 
the practices. These are the kinds of decisions made by people who call 
themselves behavioral technologists in education. 

This program is going to be piloted before it's to be implemented 
and there will be a lot of feedback on the decisions we made. There 
will probably be some further jockeying around before the course is 
finally implemented, but our best guess is that at this point we have the 
most efficient and effective structure that we can make and it comes about 
from having done particular analysis of job tasks. 

We've done the same kind of thing for sales training, the area of 
teaching salesmen for a large, multi-division company to sell. The same 
type of analysis was done for the ATT first-aid project Just completed, 
which shrank first-aid training for ATT from a day and a half to hours. 
Right now we're doing a project for hospitals and trying to develop a 
curriculum for training surgical, technicians, scrub nurses, and people 
in the operating rooms. I have a brochure here that discusses it ev;*;r so 
briefly and is, in fact, pictoral evidence of what we do. 
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ATT had the same dollar problem which la how we prefer to work with 
problems. ATT is the only company in the nation to do their own first- 
aid training and to certify instructors. The problem was they trained 
3000 people a year in firs 0 : aid and it took about lj weeks. They finally 
cut it down to 10 hours. They said 10 h mru was as useless as 16 hours 
because that's another day off the job. They estimated the cost and if 
they could get the first-aid training down to less than a day and a half- 
million dollars would be saved in a year. With that they gave half that 
to American Institutes for Research to do something with and it's a 
beautiful project. Basically what AIR did was design the performance 
objectives and a way to evaluate performance. They gave that test to 
people who had been trained in the other system and to the people who 
had gone through the new system. They did not get any overlap. 



APPENDIX G 



TALENT INVENTORY 
IDENTIFICATION 

■ % 

Statement oi the Problem 

The construction of a massive curriculum research, development and dissem- 
ination project involving the total field of business Education (such as NOBEL?) 
required the identification of appropriate personnel. Personnel that should be 
involved. No comprehensive data were available on the number or talents of 
potential participants. Consequently the identification of this potential talent 
was made a Dart of Moonshot. The Specific Procedures in the Moonshot feasibility 
proposal discussed the three facets involved in identifying appropriate 
personnel: 

A. Development of a roster of selected personnel: state supervisors, city 

supervisors, ity supervisors, business teachers educators, and others. 

B. Securing talent data by means of a mailed instrument; (see pp. 65 - 66 ). 

i 

C. Development of a computer program for analyses of the talent available 
for NOBELS. (See page 68 for a sample printout). 

Significance 

Curriculum innovation must have the support of the decision makers within the 
effected discipline. A calculated procedure for securing such support is the 
direct involvement of these decision makers in the proposed curriculum project. 
Furthermore, the proposed Talent Inventory of appropriate decision makers would 
provide the additional advantage of a vehicle for dissemination of information 
concerning the project at each phase. Finally, it would provide comprehensive 
directory of the talent currently available for other professional projects. 

Purpose 

The major purpose of the Talent Inventory Sub-contract was to determine 
potential participants in any future phases of NOBELS as well as their unique 
talents and willingness to participate. The specific objectives of this purpose 
are: 

1. Compile an initial list of potential participants. 

2. Develop a talent data collecting instrument. 

3. Develop sources for the collection of talent data. 

4. Develop procedures for processing and analyzing data. 

5. Develop a directory of personnel available and/or interested in working 
on the project. 

Report to the Delta Pi Epsilon Board of Governors and other appropriate 
agencies on the results of Part 1 of the talent operation. 
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Methods and Procedures 



For each specific purpose, a specific procedures was utilized. These were: 

1. The initial lists of potential participants were secured from the 00-TEC 
(Office Occupations -Teacher Education Clinics) files. 1 

2. Key punch mailing address and code by level (business Teacher Educuators, 
State Supervisors, City Supervisors). 

3. Machine run by institution within the state. 

4. Write an explanatory memorandum about the project to the state supervisors 
requesting data on appropriate personnel m their state. 

5. Mail the complete run by four-year degree granting instututions within the 
state to the respective state supervisors for additions and deletions of 
those institutions certifying high school business teachers. 

6. Write cover memorandum about NOBELS to these institutions to secure: 

a. additions 

b. deletions 

c. identification of the department chairman 

d. doctoral program, and if so, a list of their doctoral students 

e. other potential participants at their respective institutions. 



PROCEDURES 



1 . 



2 . 



b 

5 * 

b 

f 



3. Because of the limitations of time and cost, the total identification 
project was divided into the following phases: 

PHASE I 

Field test in Michigan 
Refine and use in Michigan 



1 Project #6-1522-l-32U. Department of Health, Education and Welfare, Office 
of Education, Bureau of Research, Division of Adult and Vocational Research; Project 
Director, Dr. Fred S. Cook, Wayne State University, 1966. 
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The initial lists of potential participants was secured from the 00-TEC 
Files. 

Concurrent with the development of the up-dated lists, the Talent 

Inventory instrument submitted to the USOE with the original proposal 

was field tested and revised three times. Prior to the second field 

test, the instrument was reviewed by the members of the profession at a special 

meeting. This instrument was sent to Washington, 7). C. , with a form 

SS-83 (Supporting Statement for Extra Mural Research Projects) for approval. 

Attached is a copy of the instrument. 
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PHASE II 



Refine the instrument and use in North-Central Region as defined by 
the North-Central Business Education Association. 

PHASE III 

Use the refined instrument to collect data nationally. 



The following is a brief description of each of the specific methods that were 
utilized; 

1. A mailing was sent to 54 supervisors of the fifty states, three terri- 
tories and the District of Columbia under the sanction of Mr. S. Greer, President 
of NASSBOE (National Association of State Supervisors of Business and Office 
Education). The mailing contained the names of all known schools with business 
teacher education programs, the names of Business Teacher Educators, as well as 
State and City Supervisors of business and office education. The respondents were 
asked to make additions, deletions, and to add the names of any schools not listed 
on the printout which certified one or more business teachers per year. While 
the returns were somewhat delayed, the response was 100 percent* 

2. Concurrent with the developmnet of the up-dated lists, the Talent Inventory 
instrumnet submitted to the USOE with the original proposal was field tested and 
revised three times. Prior to the second field test it was reviewed by members 

of the profession at a special meeting. This instruemnt was sent to Washington, 

D. C., with a form SS-83 (Supporting Statement for Extra Mural Research Projects) 
for approval. Attached is a copy of the instrument. 

3. At the time returns were being received, a list of the total population 

of colleges in the United States was coded. The object of this list was to insure 
a detailed printout, if required, of this information while keeping the instrument 
data to a maximum of four punched cards. 

4. The field testing of the instrument and subsequent revision reduced 
the time required from 35 to 45 minutes to 4 to 6 minutes. This substantial 
reduction in time, while maintaining all of the pertinent questions, was 
anticipated to increase the response by at least 15 percent- The revised 
instrument was set in hot -type to produce the highest quality possible in the 
final instrument. Enve,opes were also imprinted with the word Moonshot and the 
USOE Project Number in a further attempt to increase the response. 

5. Although the Center for Research and Leadership Development in Vocational 
and Technical Education held the subcontract for Disseminat ion , it was felt 

that a greater response would be achieved if materials were included with the 
Talent Instrument . Therefore, the instrument together with several articles 
published in Business Education World and the Journal of Business Education were 
sent to acquaint respondents with the project. Because it was anticipated that 
NOBELS would continue as an on-going project in Business Education, no cut-off 
date for receiving instruments was established. Consequently, late returns may 
be included in future printings of the directory. 
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, J* most difficult problem in the identification subcontract was the 

resnon^rr ° f +* ? 0rina * f ° r reportin g the data; because of the total number of 
respondents anticipated, a one-page format was not feasible. 

CONCLUSIONS 

The following conclusions were derived from the study: 

1. The returns have been 921 or 51.7 percent out of a total of 1780 mailed 
instruments. A departmental questionnaire was also included to obtain the names 
of doctoral students if the particular school had a doctoral program as well 

Prom thi P ° partlcipants at tha * school, not necessarily business education 

Prom this questionnaire, 447 new names were obtained; of this total 237 or 53 
percent of the new name returns were recieved. 

2 - Tbe ^ nal format developed is an alphabetic directory. Attached is a two- 
Ln^v^al"s ^a M appe^s'on There ** **“ “ indSX * W, ‘ ich indicateS tha pa « a th * 

las begun h on%Lse 3 II? en C ° mpleted as far as data collection is concerned. Work 



RECOMMENDATIONS 

The following recommendations are made as suggestions for an on~going 
dentification study: 

F ° r ^ he Purposes of the NOBELS Project , a one-page format must be developed 
elect or - a . c °”P^ er Program that will allow the Principal Investigator to P 

ducator thai h ln<3lVldU ^ S U3 - ng s P ecific criteria— e.g. , a Business Teacher 
! h '" 0re ha " flV6 yearS ° f tea °hing experience on the secondary 
el, with two or more years of work experience and whose major interest is in 
ne area of office and so on. 

2 * If * he NOBEJLS Project is funded it is recommended that the data from a single 
tate be updated and the procedures utilized for the national study. 

3. That an intensified effort be made to complete Phase II by January, 1969. 

4. That a preliminary directory be made available to all contractors and 
ab-contractors for their use in recruiting appropriate personnel. 

5. That tne national data collection be initiated no later than December, 1968. 

6. That as data from the other contracts in this project become available the 

£ndTf£r this^tM f °t m0re effective diss emination of appropriate findings.’ 
unds for this latter item are not part of the Identification Sub-contract! 

ZU be panted to print and distribute copies of the directory 

cost (if the University will permit such a procedure). 
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OUTLINE OF ACTIVITIES CONDUCTED BY THE CENTER FOR VOCATIONAL 
AND TECHNICAL EDUCATION ON PROJECT NOBELS 

June 28, 1967 • December 31, 1967 
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July 18: Battelle Institute 

Discussion of Systems Approach to Education 



■■t 

'•.1 



Participants 

Dr. Bill Hitt, Head Behavioral Sciences, Battelle Institute, Columbus, Ohio 
Dr, J, Marshall Hanna, Professor, Business Education, The Ohio State 
University, Columbus, Ohio 

Dr, Inez Ray Wells, Professor, Business Education, The Ohio State University, 

Columbus, Ohio 

Dr, Harry Huffman 

Mrs, Marla Peterson 

Miss Annell Lacy 

July 24: Battelle Institution 

Discussion of Planning the Research Program 

Participants 

Dr. Bill Hitt 
Dr, Harry Huffman 
Mrs, Marla Peterson 
Miss Annell Lacy 

July 26: Meeting at Battelle Institute 

Discussion of Proposal Preparation 

Participants 

Dr. Bill Hitt, Head Behavioral Sciences, Battelle Institute 
Dr, Frank Laniiam 
Dr, Harry Huffman 
Mrs, Marla Peterson 
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August 5-8: Ann Arbor* Inn America 

NOELS Proposal Writing 

Dr. Joseph Hill 
Dr. Rashid Baahshur 
Dr. fired Cook 
Dr. Harry Huffman 
Mr. John Lambert 
Nr. Tom Brocfcr 

August 12-15 1 Meeting at Ann Arbor* Inn America 

NOELS Proposal Uniting 

Dr • Joseph H i l l 
Dr. Rashid Bashshur 
Mr. Tom Brody 
Dr. Fred Cook 
Dr. Harry Huffman 
Mr. John Iambert 
Dr. Frank Lanham 
Dr. Donald Tate 
Dr. Edwin Swanson 
Miss Annell lacy 

August 22 s Meeting at Ihe Center BESC Executive Meeting and a Panel of 
State Supervisors 

(See attached sheet for participants) 

August 29: Battelle Institute 

Discussion of Tentative Research Proposal 

Participants 

Dr. Bill Hitt 
Dr. Harry Huffman 
Miss Annell Lacy 

August 29 i Meeting at Die Center 

Discussion of Progress on Planning and Feasibility Study in 
Office Education and Center* s R-le in Research Project 



Participants 

Dr. Robert E. Taylor 
Dr. Biward Morrison 
Dr. Harry Huffman 
Miss Jkinell Lacy 
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LIST OF PARTICIPANTS 
Meeting of August 22, 1967 
The Center for Vocational and Technical Education 

Columbus, Ohio 



i r. Robert D. Balthaser 
State Supervisor of Business and Office 
Educat ion 

State Department of Education 

Columbus, Ohio 

• 

Dr, Bruce Dlackstone 
Office Occupations Education 
Division of Vocational and Technical 
Education 

U . S. Office of Education 
Washington D, C* 20202 

Dr, Calfrey Calhoun 
Chairman of Business Education 
School of Education 
The University of Georgia 
Athens, 



Georgia 



Dr. Fred Cook 

Chairman of Business Education 
Wayne State University 
Detroit, Michigan 

Mrs* Dorothy Draper 

205 Cordell Hull Building 

Nashville, Tennessee 37219 

Mr* Weldon Else 
State Office Building 
Des Moines, Iowa 50319 

Mr. Larry Everett 
Research Assistant 
The University of Michigan 
Ann Arbor, Michigan 

Mr, Everett Fuller 
Director of VOE 
Texas Education Agency 
Austin, Texas 78781 

Dr. J* Marshall Hanna 
Professor of Education 
The Ohio State University 
Columbus, Ohio 

Dr* Joseph Hill 
Dean, Graduate School 
Wayne State University 
Detroit, Michigan 
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Mr* James Houstman 

State Supervisor of Office Education 
Suite "F" 

Harvey Building 

Sante Fe, New Mexico 87501 

Dr* Harry Huffman 
Specialist, Business and Office 
Education 

The Center for Vocational and Technical 
Education 

The Ohio State University 
Columbus, Ohio 

Mr* Robert E* Kozelke 
State Supervisor of Business and 
Office Education 
403 Centennial Building 
Springfield, Illinois 






Miss Annell Lacy 
Research Associate 
The Center for Vocational and 
Technical Education 
Columbus, Ohio 

Dr* Frank Lanham 
School of Education 
The University of Michigan 
Ann Arbor, Michigan 

Dr* Edward Morrison 
Research Coordinator 
The Center for Vocational and 
Technical Education 
The Ohio State University 
Columbus, Ohio 

Mrs. Marla Peterson 
Projects Coordinator 
The Center for Vocational and 
Technical Education 
The Ohio State University 
Columbus, Ohio 

Dr. Robert E. Taylor, Director 
The Center for Vocational and 
Technical Education 
The Ohio State University 
Columbus, Ohio 

Mr* Jim Wykle 
Program Officer 
U. S. Office of Education 
72 50 Seventh Street, N*E. 

Room 404 

Atlanta, Georgia 30323 
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September 12, 13* 



Meeting at The Center ifJ . - 

Discussion of Changes that Should he Made in Tentative 

Proposal 



Participants 



Dr. Fred Cook 
Dr. Frank Lanham 
Dr. Harry Huffman 
Dr. Edvard Morrison 
MLss Annell Lacy 



September 19* 



Meeting at The Center 
Methods of Analyzing Office Work 



Participants 



Dr. F 'Sward Morrison 
Dr. Harry Huffman 
Mrs. Marla Peterson 
MLss Annell lacy 



September 28: 



Meeting at The Center A _ . . . 

B»e Office as a Part of the Management information System 

(See attached sheet for participants) 



September 29* 



X^uJtZSSLt 28, 1967, "The Office as a Part of 
the Management Information Systems" 



Participants 

Dr. Charles B. Hicks 
Dr. J. Marshall Hanna 
Dr. Harry Huffman 
Miss Annell Lacy 



October 3* U, 5* 



etings at The Center n ■v.-inn- 
scussion of Methods and Procedures for Accomplishing 
^Ac-hlvea of Determining Office Activities 



Participants 

Dr. Charles Hicks 
Dr. Harry Huffman 
Mrs. Marla Peterson 
Miss Annell Lacy 
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PARTICIPANTS 



Dr. Adrian McDonough 
Wharton School of Finance and Commerce 
The University of Pennsylvania 
Philadelphia, Pennsylvania l?10li 



Mr. Richard Neumaier 
Management Consultant 
Glenside, Pennsylvania 19038 

Dr. Charles B. Hicks 
Professor, Business Organisation 
The Ohio State University 
Columbus , Ohio 

Dr. J. Marshall Hanna 
Professor, Business Education 
The Ohio State Uhiversity 
Columbus, Ohio 1*3212 

Dr. F. Kendrick Bangs 
Professor of Office Management 
School of Business 
University of Colorado 
Boulder, Colorado 80302 

Dr. Harry Huffman 

Specialist, Business and Office Education 

The Center for Vocational and Technical Education 

Columbus, Ohio U3212 

Mr. Clyde Welter 
Research Associate 

The Center for Vocational and Technical Education 
Columbus , Ohio 2*3212 

MLss Annell lacy 
Research Associate 

The Center for Vocational and Technical Education 
Columbus , Chio 1*3212 
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October 10; Meeting at The Center 

Alternative Approaches for Looking at Office Activities 

and Writing Behavioral Objectives 



Partic Ipants 

Dr. Charles Hicks, Professor, Business Organisations, The Ohio 
State tfaiversity, Columbia#, Ohio 
Dr. Prank Lanham 
Dr. Pred Cook 
Dr. Edward Morrison 
Dr. Harry Huffman 
Mrs. Marla Peterson 
Miss Annell Lacy 
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APPENDIX H-2 



REPORT OF ACTIVITIES CONDUCTED BY THE 
CENTER ON PROJECT NOBELS 

June 28, 1967 - December 31, 1967 



The first meeting which the Center staff had during the current 
period was at Battel ie Memorial Institute on July 18. Dr. William IHtt, 
Director of Behavioral Sciences at Battelie, was asked tu explain the 
systems approach to education. 

Dr. Hitt described the systems approach as it has been used in rela- 
tion to other fields of study. Some of the points made concerning the 
systems approach to education were* 

1. The systems approach is trying to optimize resources— in order to do 
what is best for the particular system there must be a number of vari- 
ables to consider. 

2. Objectives must be set up which can be achieved with available re- 
sources, and those restraints which might prohibit the accomplish- 
ment of stated objectives must be indicated, 

j. Alternative strategies must be determined. As the research project 
progresses, the researchers must come up with alternative strategies 
in getting at the problem. 

4. Establish an attitude for looking at the problem. The researcher must 
keep in mind that there is no one procedure for researching the prob- 
lem. Objectives must be related and looked at in operational terms. 
There must be an hierarchy of objectives - an operational level. 

5. Method of attaching the research project. 

1. Establish content elements to meet each objective. 

2. Research objectives as to cost and other benefits. 

3. Determine alternative methods. 

On July 24 the Business and Office Staff at the Center met with Dr. 
William Hitt, Director of Behavioral Sciences, Battelie, to discuss 
"Planning the Research Program. 1 ' 

The following items were discussed in relation to planning the re- 
search program. 

1. What are the objectives of the research program? 

2. What is the relative importance of these objectives? 

3. How can the accomplishment of these objectives be measured? 
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4. What are the resources available to the research program? 

5. What are the boundary conditions imposed on the research program? 

6. What are reasonable alternative strategies for accomplishing the 
program objectives? 

7. What are the potential benefits associated with each alternative 
strategy? 

8. What are the estimated ’’costs” associated with each alternative/ 
y. What alternative strategy should be selected? 

10. What steps should be taken in implementing this strategy? 

This meeting served as a background for planning the meeting held in 
Chicago on June 30, 31, 1967. 

The Center Staff consulted with Dr. Bill Hitt on July 26 to discuss 
proposal preparation. The following items were discussed: 

1. What is the system under study? 

a. What are the objectives of the system? 

b. How can the accomplishment of these objectives be measured? 

c. What are the components of the system? 

d. What parameters influence the system? 

e. What are the important properties of the system? 

2. What is the problem? 

a. What is the present state of the system? 

b. What is the desired state of the system? 

c. What is the nature of the discrepancies between the actual and 

the desired? 

d. Why do these discrepancies exist? 

3. What should be done? 

a. What are alternative strategies for solving the problem? 

b. What are the pros and cons of each alternative? 

c. What appears to be the most reasonable alternative? 

L, 'That can be expected? 

a. Hov is the system likely to change as a result of implementing 
the results of the selected alternatives? 

, What, progress indicators can be used to assess progress? 
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The BESC Executive Committee invited consultants from other dis- 
ciplines to a meeting in Chicago on July 30 and 31. At this meeting the 
progress of the research project was presented to the participants in 
order to bring them up to date. After the nature of the project had been 
described , the consultants made suggestions and recommendations to the 
BESC Executive Committee. These suggestions and recommendations included 
ways of incorporating into the Business and Office Project research that 
has been done in other disciplines. 

On August 5-8 and August 12-15, members from the BESC Executive Conw 
mittee met at Ann Arbor, Michigan, to draw up a tentative proposal for 
researching behavioral objectives for business and office education. 

State supervisors met with the BESC Executive Committee on August 22 
at the Center. This meeting focused on problems and suggestions which 
the state supervisors thought significant to research to determine the 
behavioral objectives for business and office education. 

The BESC Executive Committee asked for suggestions from the state 
supervisors as to how the state supervisors would like to help partici- 
pate and cooperate in the total research project. 
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Following the siting of the tentative proposal, thr Center staff 
n r t with Dr. William Hitt, Battclle, on August 29 to evaluate thr pro- 
posal. The primary objective in meeting with Dr. Hitt was to deterring 
whether the systems approach had been consistently and accurately writ- 
ten Into the proposal. Some of the comments which Dr. Hitt made were: 

1. Define the system which you are talking about more clearly. 

2 . Diagram the total program to indicate how various projects are 
interrelated. 

3. Define business and office education. 

4. Indicate predictions for business and office education 

a. Office requirements: manpower 

b. Office equipment: changes in equipment over the next 10 years, etc. 

5. Indicate how this approach differs from other approaches. Show what 
is really new in the project. 

6. Specify what the end product is expected to be. 

a. Is it guidelines? 

b. Is it a technique? 

c fc Is it tested materials? 

7. What difficulties do you expect to encounter? 

a. Adoption problems 

b. Predictions of future requirements 

c. Implementation 

The Center staff in Business and Office Education met with the Direc- 
tor of the Center, Dr. Robert Taylor, after the meeting with Dr. Hitt on 
August 29. The meeting with Dr. Taylor was to determine what the Center’s 
role should be in the project. Dr. Taylor suggested the following involve- 
ment for the Center: 

1. Coordination of research 

2. Communication of project progress 

3. Long-range planning - look not only for today’s problems, but tomor- 
row’s problems as well 

4. Research Management — A research management expert should audit re- 
search at key points 

5. General Support - provide training sessions, project staff for train- 
ing of research techniques, provide continuing consultation. 

A telephone survey of business firms in Columbus was made on Septem- 
ber 6, to determine the extent of familiarity of personal managers with 
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the Dictionary of Occupational Titles and Its uses. The survey of several 
large firms in Columbus, indicated that only two personnel managers were 
familiar with the DOT and that only one of the firms actually used the 
DOT as a guide for establishing titles for office workers. 

A meeting was held at the Center on September 12 and 13, to deter- 
mine the changes that should be made in the tentative proposal which was 
written during August at Ann Arbor, Dr. Huffman, Dr. Lanham, and Dr. 

Cook discussed and analyzed the changes that had been suggested to them 
by various consultants since the time the proposal had been written. 

Information management consultants were invited to the Center on 
September 28 to discuss the role of th office in the handling of in- 
formation. The following is a report of the outcomes of that meeting. 

After the information management consultants had met with the Center 
Staff, a meeting was held on September 29 to synthesize the comments of 
the information management consultants. The synthesis was then used as 
a basis for planning research of the office and its place in facilitating 
information. 

Writing sessions were held at the Center on October 3, 4, and 5, to 
outline the methods and procedures for determining the activities of the 
office in handling information. Consultants from business education 
and management helped write the methods and procedures. 

After the Center staff had completed the outline for researching 
office activities as they relate to information handling, a meeting was 
held with other BESC members at the Center on October 10 to determine 
alternative approaches for looking at office activities and writing be- 
havioral objectives. 

During the months of November and December the Center staff made 
a search of secondary sources for office activities. The activities 
that were secured from secondary sources will be used to make a taxon- 
omy of office activities. The taxonomy of office activities will then 
be used as a basis for analyzing office jobs. After the analysis of 
office jobs has been completed the taxonomy will be rewritten to in- 
clude additional activities that have been determined from the job 
analyses. 
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A Research Planning Meeting 
September 28, 1967 

The Office as a Part of the Management Information System 



The primary purpose of this meeting was to identify way* of deter- 
mining what is going on in the office now and in the future. A know- 
ledge of what is going on in the office is needed as a basis for dovelop- 
ing°prograns and curriculum materials to train people for office work. 

Overview 

Each year the number of productive blue-collar workers decreases 
while the number of white-collar woakers increases. In a society where 
the work of white-collar workers acccounts for approximately 50 percent 
of the gross national product, the white-collar workers must be consider- 
ed productive. The question is, "What is the white-collar worker pro- 
ducing?" 

Many office management experts suggest that the final product of 
the white-collar worker is information. The raw material of the office, 
that is, information, is data, and the process through which the raw 
material must go to become the final product of information is data pro- 
cessing. Data that has been processed becomes inventory until it is 
needed for decision-making within the organization, and then it is in- 
formation. 

Office workers are responsible for the processing of data. There- 
fore, leaders in business and office education are seeking ways to deter- 
mine all the activities that are involved in processing data in order to 
prepare young people for office work. 

Area of Concern 

The participants discussed several areas that are of vital concern in 
determining the role of the office worker in the processing of data and 
the handling of information. Some of the questions raised and a brief 
analysis of the discussion concerning each question follows: 

1. What kinds of information are needed? 

Basically, there are two needs for information: (a) routine pro- 

cessing of clerical papers i.e,, invoices, paychecks and (b) re- 
ports to aid management decisions. 

2. Should office workers be trained in terms of automated offices? 



Statistics show that 99 percent of all firms employ less than 20 
office workers. Yet, probably 99 percent of all office workers 
are employed in large corporations that employ 20 or more office 
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workers. Therefore, the number of office workers in th f nen- 

cornputer office may be relatively few in number. Those 

that do not have computers may use a computer fme-sharin^ plan, 

Tl hat is the ultimate objective of office work? 

Increased cost of hardware equipment have resulted in an in- 
crease in the cost of processing data. However, the ultimate 
objective in office is not one of cost reduction alone but also 
of increased productivity. The entire operation of the organi- 
zation is based on the management’s ability to have relevant in- 
formation at ait times. The increase in productivity of salable 
services will sometimes increase overall profits of an organiza- 
tion to the extent that hardware costs are offset. 

How does one determine what data is relevant? 

Many organizations have more data accumulated than is relevant 
or needed for the operation of the organization. This can be 
thought of as excessive inventory. Office managers need to know 
how to determine what data and information is relevant to the 
organization's needs. 

Do we train students for terminal jobs? 

In today's constantly changing technological society, workers 
cannot assume that they will enter a job and remain in it. 
Therefore, training is considered terminal only to the extent 
that students will stay on in the world of work rather than re- 
turning to school. 

What are the fundamentals that we want from the high school 
graduate? 

Students may be trained in terms of manipulative skills or taught 
specific knowledges. Office workers today must have accuracy, 
logic, and skill. Certain decision-making activities are re- 
quired of office workers in most jobs. One method of determin- 
ing the skills an knowledges required might be through an an- 
alysis of mental and analytical activities in the office. 

How does one determine which jobs to analyze? 

There are various lists of job title classifications which could 
be used. For example, the DOT put out by the U. S, Department of 
Labor, the list put out by AMS, and many large corporations have 
their own job title classifications for ail their branch offices. 
Using job titles are often misleading. In the past many job 
ti ties have been based on pay rather than job content. Perhaps 
job titles as they are currently used in offices could be used 
only as a guide to determine office jobs to analyze. These of- 
fice jobs could then be analyzed to determine activities that ac- 
tually take place in the office. The activities could then be 
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grouped into clusters of common act r vities. The c 1 s c y 
activities could presumable fall into clusters of comonal - t v 
in areas of work as well as in areas of subject matte' - . 

After job activities have been determined, how can one d*- tr rm* tv 
whether or not to include such activities in an educational pro- 
gram? 

The following criteria might be used Cor including or el irern ti nr 



act 


ivi ties ; 




a) 


Frequency 




b) 


Difficulty 




c) 


Universal ity < 


of use 


d) 


Most feasible 


place 


e) 


Importance of 


ar ti v 


What are some of 


the th 


a) 


Meehan izati on 




b) 


Hardware 




c) 


Space 




d) 


Furniture 





e) Data communications 

10. Can students be taught in such a way that there is a transfer 
of learning from the classroom situation to the office? 

Experience indicates that workers under 20 years of age are less 
adaptive, to new situations and change than are workers 10 years 
or more of age. Perhaps, it would be fruitful to determine the 
elements of adapting to new situations and determine whether they 
can be taught. 

The ability to transfer learning is highly dependent on the 
worker* s attitude. Some of the attitudes which should be develop- 
ed are: 



a) Willingness to change 

b) Attention to detail 

c) Discipline to accuracy 

d) Timeliness (Promptness) 

e) Curiosity to learn. 

The participants of this meeting were in agreement that a method for 
analyzing job activities and the criteria for selecting the offices to be 
used in the analysis is of primary importance. 

The following list of procedures for determining office job activities 
might be used: 

a) Develop a job analysis instrument 

b) Select the office sample 

c) Analyze jobs for activities 

d) Evaluate activities 
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r) Group activities into clusters 

f) Prepare classification of job-activity clusters. 

11. ’.'hat is a model for examining some of the problems in office educa- 
tion? 

Areas of Influence on Office Education (A Model) 



Shaded part represents what we know, and therefore represents the 
limist as to what can be taught in these eight segments. 

12. What is a list of verbs that describe office activities? (The be 
ginnings of such a list follows.) 




Information 



Office Skills 
(Activity verbs) 
^ee list below^ 



Communications 



Data 



Applications 
(Nouns ) 



Office Hardwar 
Development 



Conditions 



(Modi f i er ) 



/ and attitude 



level 



Filing 

Sorting 

Classifying 

Coding 

Indexing 

Releasing 

Abstracting 

Computing 

Comparing 

Dictating 

Recording 

Evaluating 

Graphing 

Testing 

Data Reducing 



Retrieving 

Matching 

Inspecting 

Extending 

Con tro ling 

Flow-charting 

Analyzing 

Corresponding 

Matching 

Transcribing 

Duplicating 

Inferring 

Planning 

Communicating 

Sequencing 



Conversing 

Typewriting 

Table searching 

Calculating 

Keypunching 

Updating 

Judging 

Data Accumulating 

Data Transferring 

Documenting 

Retaining 

Charting 

Programming 

Reporting 

Allocating 
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PARTICIPANTS 



Dr. Adrian McDonough 
Vlharton School of Finance and Commerce 
The University of Pennsylvania 
Philadelphia, Pennsylvania 1910k 

Mr. Richard Neumaier 
Management Consultant 
Glenside, Pennsylvania 1903 S 

Dr. Charles B. Hicks 
Professor, Business Organization 
The Ohio State University 
Columbus, Ohio 1*3212 

Dr. J Marshall Hanna 
Professor, Business Education 
The Ohio State Univ.3 rsity 
Columbus, Ohio 1*3212 

Dr. Kendrick Bangs 
Professor of Office Management 
School of Business 
University of Colorado 
Boulder, Colorado 80302 

Dr. Harry Huffman 

Specialist, Business and Office Education 

The Center for Vocational and Technical Education 

Columbus, Ohio 1*3212 

Mr. (Hyde Welter 
Research Associate 

The Center for Vocational and Technical Education 
Columbus , Ohio 1*3212 

Miss Anne 11 Lacy 
Research Associate 

The Center for Vocational and Technical Education 
Columbus , Ohio 1*3212 
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